Use this page after reading the derivations. Every formula is attached to the conditions that make it useful.
1. Notation
| Symbol | Meaning |
|---|---|
| Y, X | Population random variables |
| \(Y_{i}, X_{i}\) | Random observation i; lowercase symbols may denote realizations |
| \(\mu , \sigma ^{2}, \sigma , \sigma _{XY}, \rho _{XY}\) | Population mean, variance, SD, covariance, correlation |
| \(\bar{Y} , s^{2}, s, s_{XY}, r_{XY}\) | Sample analogues; random before the sample is observed |
| \(\hat{\theta } , \hat{\delta } , \hat{p}\) | Estimator of a general parameter, a difference, a proportion |
| \(\operatorname{SE}(\hat{\theta } )\) | Estimated sampling standard deviation of \(\hat{\theta }\) |
| \(\xrightarrow{p} , \xrightarrow{d}\) | Convergence in probability; convergence in distribution |
| \(\Phi , z_{q}\) | Standard normal CDF; its q-quantile |
| \(\mathbb{E}*, \operatorname{Var}*, \hat{\theta } *\) | Conditional bootstrap expectation, variance, and estimate |
2. Formula sheet
| Result | Formula | Condition / module |
|---|---|---|
| Expectation of a function | \(\mathbb{E}[g(Y)]=\sum g(y)p(y)\) | Integrability; 02 |
| Variance identity | \(\operatorname{Var}(Y)=\mathbb{E}[Y^{2}]-\mathbb{E}[Y]^{2}\) | Finite second moment; 02 |
| Iterated expectations | \(\mathbb{E}[Y]=\mathbb{E}[\mathbb{E}[Y\mid X]]\) | \(\mathbb{E}|Y|<\infty\); 03 |
| Total variance | \(\operatorname{Var}(Y)=\mathbb{E}[\operatorname{Var}(Y|X)]+\operatorname{Var}(\mathbb{E}[Y|X])\) | Finite second moment; 03 |
| Variance of a difference | \(\operatorname{Var}(X-Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)-2Cov(X,Y)\) | Finite variances; 04 |
| Mean sampling variance | \(\operatorname{Var}(\bar{Y} )=\sigma ^{2}/n\) | i.i.d., finite variance; 06 |
| LLN | \(\bar{Y} \xrightarrow{p} \mu\) | i.i.d., finite variance suffices; 06 |
| CLT | \(\sqrt{n}(\bar{Y} -\mu )/\sigma \xrightarrow{d} \mathcal{N}(0,1)\) | \(i.i.d., 0<\sigma ^{2}<\infty\); 06 |
| MSE | \(\operatorname{Var}(\hat{\theta } )+\operatorname{Bias}(\hat{\theta } )^{2}\) | Finite second error moment; 07 |
| Sample variance | \(s^{2}=\sum (Y_{i}-\bar{Y} )^{2}/(n-1)\) | \(n\ge 2\); unbiased under i.i.d.; 08 |
| Mean SE | \(s/\sqrt{n}\) | i.i.d. model; 08 |
| Two-sided p-value | \(2[1-\Phi (|t|)]\) | Normal reference justified; 09 |
| Mean confidence interval | \(\bar{Y} \pm z_{1-\alpha /2}\operatorname{SE}\) | Large-sample route; 09 |
| Independent-group SE | \(\sqrt{s_{A}^{2}/n_{A}+s_{B}^{2}/n_{B}}\) | Independent groups; 10 |
| Paired SE | \(s_{D}/\sqrt{n}\) | Independent pairs; 10 |
| Exact mean interval | \(\bar{Y} \pm t_{n-1,1-\alpha /2}s/\sqrt{n}\) | i.i.d. normal population; 11 |
| Sample correlation | \(s_{XY}/(s_{X}s_{Y})\) | Positive sample SDs; 12 |
| OLS slope with intercept | \(\hat{b} =s_{XY}/s_{X}^{2}\) | Nonconstant sample \(X\); 12 |
3. Common mistakes and repairs
| Incorrect shortcut | Correct distinction |
|---|---|
| “A density height is a probability.” | Continuous probabilities are areas. |
| “\(\mathbb{E}[g(Y)]=g(\mathbb{E}[Y])\).” | Linearity works for affine g; nonlinear transformations need separate calculations. |
| “Zero correlation means independence.” | It removes linear association; joint normality is a special exception. |
| “The CLT makes observations normal.” | It approximates a centered and scaled average. |
| “An unbiased estimator must be consistent.” | \(Y_{1}\) is a counterexample. |
| “SE is \(s/n\).” | For the i.i.d. sample mean it is \(s/\sqrt{n}\). |
| “A t-statistic always has a t distribution.” | Exact \(t\) requires distributional assumptions. |
| “\(p=.04\) means a 4% chance \(H_{0}\) is true.” | It is a tail probability computed under \(H_{0}\). |
| “95% confidence covers 95% of people.” | An interval for a mean describes estimation uncertainty. |
| “Subtract standard errors for a difference.” | Combine variances, including covariance when needed, then take a square root. |
| “Significant in one group but not another proves a difference.” | Test the difference directly. |
| “A precise observational gap proves causality.” | Identification and sampling precision are separate. |
| “More bootstrap draws fix invalid bootstrap inference.” | They reduce Monte Carlo error only. |