Table of Contents

Module 4: Statistics & Probability

Continuing the Data Analyst track — Statistics module. This is the section where interviews separate “can run a t-test in a library” from “understands what the test is actually claiming,” so I’ll lean into the conceptual traps and the talking points that show you understand the assumptions, not just the formulas.

1. Descriptive Statistics — Mean/Median/Mode, Variance, Distributions

Mean vs. Median vs. Mode — the real interview question is “which one lies to you and when”:

Measure	Sensitive to outliers?	Use when
Mean	Yes — heavily	Symmetric distributions, when total/sum matters (e.g., total revenue ÷ count)
Median	No — robust	Skewed distributions (income, latency, house prices) — the canonical “average household income” should almost always be median, not mean
Mode	No	Categorical data, or identifying the most common value in a multimodal distribution

Interview talking point: “If someone reports ‘average revenue per user’ without telling me the distribution shape, my first question is whether it’s mean or median — a handful of whale accounts can make the mean wildly unrepresentative of a typical user. I’ve seen exec dashboards report mean ARPU and accidentally hide a churn problem in the median user, because growth in a few large accounts masked decline everywhere else.” This is a real, commonly-cited business scenario and signals you think about metric design, not just computation.

Variance and standard deviation:

Variance (population): σ² = Σ(x - μ)² / N
Variance (sample):      s² = Σ(x - x̄)² / (n-1)        ← note the n-1, not n

The n-1 nuance is a frequently-tested detail: dividing by n-1 (Bessel’s correction) instead of n corrects for the fact that a sample’s mean is itself an estimate, which makes the naive variance calculation systematically underestimate the true population variance. This is exactly the kind of “why,” not just “what,” that distinguishes a senior answer.

Distribution shape — skewness and kurtosis:

Right-skewed (positive skew): long tail to the right — mean > median. Income, transaction amounts, latency.
Left-skewed (negative skew): long tail to the left — mean < median. Exam scores near a ceiling, time-to-failure in well-maintained equipment.
Kurtosis: how “fat-tailed” a distribution is relative to normal — high kurtosis means more extreme outliers than a normal distribution would predict (relevant in financial risk modeling — fat tails are why “this was a 1-in-10,000-year event” claims are often wrong).

2. Probability Fundamentals, Conditional Probability, Bayes’ Theorem

Conditional probability:

P(A|B) = P(A ∩ B) / P(B)

Read as “probability of A, given that B has already happened” — the denominator re-scopes the universe to only the cases where B is true.

Bayes’ Theorem — the formula and, more importantly, the business framing:

P(A|B) = P(B|A) × P(A) / P(B)

This is one of the highest-value interview topics because it’s almost always tested through the classic false-positive trap, which every analyst should be able to walk through cold:

“A test for a rare disease (1% prevalence) is 95% accurate. A patient tests positive. What’s the probability they actually have the disease?”

P(Disease) = 0.01
P(Positive | Disease) = 0.95
P(Positive | No Disease) = 0.05    (false positive rate)

P(Positive) = P(Positive|Disease)×P(Disease) + P(Positive|No Disease)×P(No Disease)
            = 0.95×0.01 + 0.05×0.99
            = 0.0095 + 0.0495 = 0.059

P(Disease | Positive) = (0.95 × 0.01) / 0.059 ≈ 0.161  →  only ~16%

Interview talking point: “The counterintuitive result — only a 16% chance of actually having the disease despite a ‘positive’ result from a 95%-accurate test — is entirely driven by the low base rate. This isn’t just a stats puzzle; it’s the exact same reasoning that applies to fraud detection, spam filtering, and churn-prediction alerting: when the event you’re detecting is rare, even a strong classifier produces mostly false positives in absolute terms, and any alerting system needs to account for base rate, not just model accuracy.” That’s a genuinely valuable bridge from a textbook problem to production ML/analytics reasoning — exactly the kind of synthesis an architect-level answer should make.

3. Hypothesis Testing — Null/Alternative, p-values, Significance, Type I/II Errors

The framework:

H₀ (null hypothesis): the “no effect / no difference” baseline assumption — e.g., “the new checkout flow has no effect on conversion rate.”
H₁ (alternative hypothesis): what you’re trying to find evidence for — e.g., “the new checkout flow changes conversion rate.”
You never “prove” H₀ — you either reject H₀ or fail to reject H₀. This phrasing precision is tested in interviews; saying “we proved there’s no effect” is a flag for a weak statistical foundation.

p-value — the most commonly misunderstood concept in all of analytics, and a near-guaranteed interview question:

A p-value is the probability of observing data at least as extreme as what you saw, assuming the null hypothesis is true. It is not the probability that the null hypothesis is true, and it is not the probability the result is “due to chance” in some informal sense.

Talking point that directly answers the most common misconception question: “A p-value of 0.03 doesn’t mean there’s a 3% chance the result is random — it means that if there really were no effect, you’d see data this extreme or more extreme only 3% of the time. People conflate P(data | H₀) with P(H₀ | data) — that’s actually the exact same logical inversion Bayes’ theorem corrects for, which is why p-values and Bayesian reasoning get debated together so often.” (Nice explicit callback connecting sections 2 and 3 — interviewers respond well to candidates who show the topics aren’t isolated.)

Type I vs. Type II errors — the confusion matrix of hypothesis testing:

	H₀ is actually True	H₀ is actually False
Reject H₀	Type I Error (false positive) — probability = α	Correct decision (true positive)
Fail to reject H₀	Correct decision (true negative)	Type II Error (false negative) — probability = β

α (significance level), conventionally 0.05, is a chosen threshold, not a law of nature — it’s the rate of false positives you’re willing to tolerate.
Power (1 – β) is the probability of correctly detecting a real effect when one exists — driven by sample size, effect size, and variance.

Talking point on the α=0.05 convention: “0.05 is a convention, not a universal truth — in A/B testing at high traffic/high-stakes contexts (e.g., a checkout flow change touching millions of users), I’d want a stricter threshold or a sequential testing correction, because running many simultaneous tests at α=0.05 inflates the overall false-positive rate across the test suite — the multiple comparisons problem. That’s solved with corrections like Bonferroni or, more commonly in practice, false discovery rate control.”

4. Confidence Intervals

CI = point_estimate ± (critical_value × standard_error)

For a 95% CI on a sample mean:

CI = x̄ ± 1.96 × (s / √n)

The interpretation trap — tested almost every time this topic comes up:

A 95% confidence interval does not mean “there’s a 95% probability the true parameter is in this interval.” It means: if you repeated this sampling process many times, 95% of the intervals constructed this way would contain the true population parameter.

Talking point: “The true parameter is fixed — it’s either in this specific interval or it isn’t; the 95% describes the long-run reliability of the method, not the probability for this one interval. I treat a confidence interval mainly as a way to communicate the precision of an estimate to stakeholders — a wide CI on a conversion lift means ‘don’t make a big decision off this number yet,’ regardless of whether the point estimate looks exciting.” This last sentence is the kind of practical business-translation point that’s worth volunteering — it shows you use CIs for decision-making, not just reporting.

Width drivers — narrower CI comes from: larger sample size (√n in the denominator — so 4x the sample only halves the CI width, a commonly tested square-root relationship), lower variance in the underlying data, or a lower confidence level (90% CI is narrower than 99% CI — there’s an explicit precision/confidence trade-off).

5. Correlation vs. Causation

The single most over-tested but genuinely important concept in analytics interviews — because it’s the difference between an analyst who reports numbers and one who’s trusted with decisions.

Why correlation ≠ causation — the standard failure modes to name:

Confounding variable — a third factor drives both. Classic: ice cream sales correlate with drowning deaths — both driven by hot weather, no causal link between them.
Reverse causation — direction is backwards. Companies with more customer support staff have more complaints — is support causing complaints, or do more complaints justify hiring more support staff?
Selection bias — the sample itself isn’t representative, manufacturing a spurious relationship.
Coincidence / spurious correlation at scale — with enough variables tested, some will correlate by pure chance (this connects directly back to the multiple-comparisons problem from section 3).

What actually establishes causation: randomized controlled experiments (A/B tests) are the gold standard because random assignment breaks the link between the treatment and any confounder. When a true experiment isn’t possible, quasi-experimental methods exist — difference-in-differences, instrumental variables, regression discontinuity — which are advanced but worth name-dropping to show awareness beyond “correlation isn’t causation” as a slogan.

Interview talking point: “Whenever a stakeholder shows me a correlation and asks ‘should we invest more in X,’ my first move is to ask whether this came from an experiment or observational data. If it’s observational, my job is to identify the most plausible confounder before recommending action — and if the decision is high-stakes enough, I’d push for a proper randomized test rather than act on the correlation alone.”

6. Common Distributions — Normal, Binomial, Poisson

Distribution	Models	Key parameters	Real analyst example
Normal	Continuous data clustering around a mean, symmetric	μ (mean), σ (std dev)	Heights, measurement errors, and — via the Central Limit Theorem — the sampling distribution of the mean for almost any underlying distribution at large enough n
Binomial	Number of successes in a fixed number of independent yes/no trials	n (trials), p (success probability)	Conversion: “out of 1,000 visitors, how many purchase,” click-through rate modeling
Poisson	Number of events in a fixed time/space interval, when events are rare and independent	λ (average rate)	Customer support tickets per hour, website errors per minute, defects per batch

The Central Limit Theorem is the connective tissue that makes all of the hypothesis-testing math in section 3 work even when the underlying data isn’t normal: “Even if individual transaction amounts are heavily right-skewed, the distribution of the sample mean across repeated samples approaches normal as sample size grows — that’s the entire justification for using normal-based confidence intervals and z-tests on business metrics that are almost never normally distributed at the individual-observation level.” This is a strong unifying point to make in an interview — it shows you understand why the standard toolkit is valid, not just that it exists.

Binomial vs. Poisson — the distinguishing question interviewers ask: “Binomial needs a known, fixed number of trials (n) — Poisson is for counting events over a continuous interval where there’s no natural ‘n,’ just a rate. ‘Will this user convert’ is binomial (each user is a trial); ‘how many support tickets arrive per hour’ is Poisson (there’s no fixed number of ‘attempts,’ just a rate of occurrence).”