A/B Test Sample Size and the Peeking Problem: Interview Guide
Size an A/B test from baseline, MDE, alpha and power, turn it into days, and see by simulation why stopping at the first p below 0.05 fails.
Prepare with explained interview questions and study notes for JavaScript, TypeScript, React, Angular, Java, Python, SQL, AWS and more, organized by topic and with the reasoning written in the margins. Finally, practice them like it's the real interview.
function outer() { let count = 0; return function inner() { return ++count; }; } const next = outer(); next(); // 1 next(); // 2 ← why not 1?!
Closures, hoisting, prototypes & the event loop.
Generics, narrowing, utility & mapped types.
Hooks, rendering, memoization & state.
Change detection, DI, signals & RxJS.
Semantics, the box model, layout & a11y.
App Router, Server Components, caching & rendering.
Debounce, Promise.all, LRU cache… write it, run the tests.
Event loop phases, streams & modules.
OOP, collections, JVM, concurrency & streams.
IoC, auto-configuration, JPA, REST & security.
Data model, generators, decorators & the GIL.
Service design, messaging, resilience & sagas.
Goroutines, channels, interfaces & errors.
Topics, partitions, consumer groups & delivery.
Big-O, arrays, trees, graphs & DP patterns.
Feeds, autocomplete, caching & performance.
HTTP, DNS, TCP and request troubleshooting.
Processes, threads, memory and concurrency.
Test strategy, isolation, edge cases and debugging.
STAR stories, collaboration and project discussions.
Bias-variance, metrics, trees & ensembles.
Backprop, optimizers, CNNs & transformers.
RAG, embeddings, agents, evals & guardrails.
Pipelines, serving, monitoring & drift.
Probability, A/B tests & hypothesis testing.
Loading today's question…
Size an A/B test from baseline, MDE, alpha and power, turn it into days, and see by simulation why stopping at the first p below 0.05 fails.
Solve the classic 1% disease test question three ways, see why rare events wreck precision, and avoid the base rate traps interviewers set.
What the central limit theorem really promises, shown by simulation on skewed revenue data, and where it quietly breaks in A/B tests.
What 95% confidence really means, how to build t, Wilson and bootstrap intervals in Python, and why the textbook proportion interval fails.
The OLS assumptions interviewers expect, which violations bias coefficients and which only break standard errors, with Python checks for each.
What a p-value actually measures, how to run a two-proportion test by hand in Python, and the misreadings that sink data science candidates.
48 must-know facts & snippets
From unknown to infer, on one page
Partial → Awaited, all of them
Rendering to Server Components, on one page
Every hook, one example each
Signals to guards, standalone-first
The 30 operators you'll actually use
Semantics to grid, cascade to stacking
App Router, RSC, caching and actions
Event loop to streams, checked on Node 22
OOP to virtual threads, on one page
Beans to JWT and @Transactional, one page
Core Python to asyncio, on one page
DDD to sagas, retries and tracing
Joins to MVCC, every query run on Postgres 16
BSON to sharding, with the defaults that matter
Goroutines to generics, slices to context
Dockerfile to Compose, layers to PID 1
Pods to Helm, plus a troubleshooting table
Data types to Cluster slots, Redis 7.4/8
EC2 to IAM to DR, the services that come up
Staging area to canary deploys, on one page
Partitions to transactions, KRaft era
GlideRecord to update sets, on one page
Big-O to DP, with Python templates
Rendering to caching, plus five mini-designs
Bayes to CUPED, p-values to regression
Bias-variance to boosting, metrics to leakage
Backprop to KV cache, Adam to LoRA
Tokens to agents, RAG to guardrails
Pipelines to PSI, registry to rollback
One tricky interview question, fully explained. Takes 5 minutes to read with your coffee.