Overall Concepts List

A one-page memory map of the language that carries Chapters 2–3. Use it to retrieve a definition, a concrete example, and the boundary where the concept stops being true.

1. Objects: what is random, and at what level?

Outcome, event, random variable

An outcome is a primitive result; an event is a set of outcomes; a random variable maps outcomes to numbers. Example: two coin tosses produce outcome HT, event “one head,” and variable \(Y=1\). An event has probability; a realized numerical value is not itself a distribution. See 01.

Parameter, estimator, estimate

\(\mu\) is a population parameter; \(\bar Y\) is the random estimator; 7.3 is one realized estimate. A standard error describes estimator uncertainty, not the spread of individual \(Y\). See 07 and 08.

Population distribution vs. sampling distribution

The first describes variation across people/items; the second describes variation of a statistic across repeated samples. If \(\operatorname{SD}(Y)=12\) and \(n=36\), then \(\operatorname{SD}(\bar Y)=2\) under i.i.d. sampling. Increasing \(n\) narrows the sampling distribution; it does not make the population less heterogeneous.

2. Probability, distributions, and moments

PMF, PDF, CDF, quantile

A PMF assigns mass to discrete values; a PDF gives density, so probability is area; the CDF \(F(c)=P(Y\le c)\) works for both; a \(q\)-quantile is a threshold reaching cumulative probability \(q\). For continuous \(Y\), \(f(y)\) may exceed 1 and \(P(Y=y)=0\); neither fact violates probability rules. Quantiles need not be unique in discrete distributions. See 01.

Expectation, variance, standard deviation, moment

Expectation is probability-weighted location; variance is expected squared deviation; SD restores the original units. Raw and central moments encode shape. Example: adding 10 changes the mean but not variance; multiplying by \(-2\) multiplies variance by 4 and SD by 2. \(\mathbb E[g(Y)]\ne g(\mathbb E[Y])\) in general, and \(\operatorname{Var}(a+bY)=b^2\operatorname{Var}(Y)\). See 02.

Joint, marginal, and conditional distribution

A joint distribution describes \((X,Y)\); summing/integrating out \(X\) yields the marginal of \(Y\); conditioning renormalizes within \(X=x\). Example: the overall employment rate is a weighted average of group-specific rates. \(P(Y\mid X)\) is a random function of \(X\); \(P(Y\mid X=x)\) is evaluated at one value. Conditioning can change both mean and variance. See 03.

Law of iterated expectations (LIE)

\(\mathbb E[Y]=\mathbb E[\mathbb E[Y\mid X]]\): average within groups, then average across groups. The outer expectation uses the distribution of \(X\), so an unweighted average of group means is generally wrong.

Conditional expectation function (CEF)

\(m(x)=\mathbb E[Y\mid X=x]\) is the minimum-MSE predictor of \(Y\) among all functions of \(X\). The residual \(u=Y-m(X)\) satisfies \(\mathbb E[u\mid X]=0\). The CEF need not be linear; zero conditional mean is stronger than zero covariance. See 03.

3. Dependence: the implication ladder

The reverse arrows fail without extra assumptions. Example: with symmetric \(X\), let \(Y=X^2\). Then \(\operatorname{Cov}(X,Y)=0\), but knowing \(X\) determines \(Y\). In a jointly normal family, zero covariance does imply independence.

Independence, mean independence, uncorrelatedness

Independence preserves the whole conditional distribution; mean independence preserves only the conditional mean; zero covariance removes only linear co-movement. “No correlation” is not “no relationship.” See 04.

Covariance and correlation

Covariance records direction and scale of linear co-movement; correlation standardizes it to \([-1,1]\). Example: changing dollars to cents changes covariance but not correlation. Correlation is unit-free but not causal, is sensitive to outliers, and may miss nonlinear dependence.

Variance of a sum and diversification

Variances add only when covariances are zero. Positively correlated assets reduce diversification; negative covariance can offset risk. Independence is sufficient, not necessary, for covariance zero; pairwise independence and mutual independence are distinct.

4. Sampling, approximation, and estimation quality

i.i.d. and simple random sampling

Identically distributed means a common marginal distribution; independent means one draw provides no information about another. Sampling without replacement from a finite population is not literally independent; clustered or time-series observations require different variance reasoning. See 06.

Law of large numbers (LLN)

The sample mean converges in probability to its population mean under suitable conditions. It justifies replacing population moments by sample analogues. LLN says concentration, not normality, and does not give the finite-sample error probability by itself.

Central limit theorem (CLT)

A standardized sum becomes approximately normal: \(\sqrt n(\bar Y-\mu)/\sigma\Rightarrow\mathcal N(0,1)\). “Large \(n\)” is distribution-dependent; skewness, heavy tails, dependence, and rare Bernoulli outcomes can slow approximation. The CLT concerns the statistic, not the raw data.

Exact vs. asymptotic distribution

Normal data give an exact Student \(t\) pivot for a mean with unknown variance; the CLT gives an approximation for broader populations. Writing \(\sim\) claims an exact distribution; \(\approx\) or \(\xrightarrow{d}\) communicates approximation or a limit. See 11.

Unbiasedness, consistency, efficiency, MSE

Unbiasedness is an expectation statement at a fixed \(n\); consistency is convergence as \(n\to\infty\); efficiency compares variability under a stated class; \(\operatorname{MSE}=\operatorname{Var}+\operatorname{Bias}^2\). An unbiased estimator can be noisy or inconsistent; a slightly biased estimator can have lower MSE. “BLUE” means best linear unbiased under its assumptions, not best among every imaginable estimator. See 07.

Standard deviation, standard error, estimated standard error

SD is a population spread; SE is the true SD of an estimator; the reported SE is usually an estimate using \(s\). The \(n-1\) divisor corrects variance-estimator bias; it is not the sample size used in \(\operatorname{SE}(\bar Y)=s/\sqrt n\). See 08.

Studentization, continuous mapping, Slutsky

Studentization divides estimation error by an estimated SD. Consistency of \(s\) plus Slutsky lets an estimated denominator replace \(\sigma\) asymptotically. A consistent denominator does not magically make a small-sample statistic exactly \(t\); exactness needs normality and the independence result.

5. Inference and causality

Null, alternative, test statistic, rejection region

The null is the benchmark restriction; the statistic measures disagreement in standard-error units; the rejection region is chosen before seeing the data. Failure to reject is not proof that the null is true. A two-sided alternative splits \(\alpha\) across two tails. See 09.

p-value, size/significance level, power

The p-value is the probability, assuming the null, of a statistic at least as incompatible with it as observed. Size is the null rejection probability; power is rejection probability under a specified alternative. A p-value is neither \(P(H_0\mid\text{data})\) nor effect size. Lowering \(\alpha\) reduces false positives but, holding \(n\) fixed, also lowers power.

Confidence level and coverage

A 95% procedure covers the fixed parameter in 95% of repeated samples under its assumptions. One realized interval either covers or does not. The frequentist statement is about the procedure, not a 95% probability assigned to the already-fixed parameter. Narrowness is not validity.

Statistical vs. economic significance

A tiny effect may be precisely estimated and statistically significant; a consequential effect may be imprecisely estimated. Report effect size, units, interval, and design. Large samples can make substantively trivial differences yield small p-values.

Independent groups vs. paired data

Independent samples use two group variances; paired analysis first forms within-pair differences \(D_i\), then performs one-sample inference. Treating paired observations as independent discards covariance and usually changes the SE. Equal group sizes do not imply equal variances. See 10.

Potential outcomes, ATE, random assignment, selection bias

Each unit has \(Y(1),Y(0)\), but only one is observed. The ATE is \(\mathbb E[Y(1)-Y(0)]\). Random assignment makes treatment independent of potential outcomes, so a difference in means identifies the ATE in expectation. A raw treated-control difference mixes treatment effects and selection when assignment is not random. Random sampling supports population generalization; random assignment supports causal identification—different jobs. See 10.

6. Regression and resampling bridges

Population regression, OLS, residual, error

The population linear projection chooses coefficients minimizing population MSE; OLS is its sample analogue. A residual \(\hat u_i\) is observable after fitting; the population error \(u_i\) is not. A fitted line is descriptive unless an identification argument gives its slope a causal interpretation. See 12.

Exogeneity and omitted-variable bias

Mean independence \(\mathbb E[u\mid X]=0\) implies zero covariance; for slope consistency, the key orthogonality is \(\mathbb E[(X-\mathbb E X)u]=0\). If an omitted cause is correlated with \(X\), the slope absorbs some of its association. Zero sample correlation between fitted residuals and included \(X\) is an OLS algebraic property; it does not prove population exogeneity.

Homoskedasticity and heteroskedasticity

Homoskedasticity means \(\operatorname{Var}(u\mid X)=\sigma^2\); heteroskedasticity lets it vary with \(X\). Heteroskedasticity need not bias OLS coefficients under exogeneity, but conventional homoskedastic SEs can be wrong.

Bootstrap and empirical distribution

The nonparametric bootstrap resamples observed cases with replacement, recomputes the statistic, and uses its conditional distribution to approximate sampling uncertainty. Resample the observational unit (pairs for regression, clusters for clustered designs). Bootstrap validity is not automatic for nonsmooth estimators, boundaries, extreme tails, or dependence. See 13.

7. Thirty-second self-check

1. Can I name the random object and its hypothetical repetitions?
2. Am I claiming equality, exact distribution, approximation, or convergence?
3. Which assumption delivers the conclusion: independence, mean independence, zero covariance, normality, or large \(n\)?
4. Does the number describe individual variation, estimator uncertainty, effect magnitude, or evidence against a null?
5. Is a comparison descriptive or causal, and what makes the counterfactual credible?