Choosing an estimator means choosing a rule for converting data into an estimate. Its quality depends on its sampling distribution and on the criterion being used.
1. Estimators, bias, consistency, and efficiency
An estimator is a function of the random sample. An estimate is its realized value. Bias and variance are finite-sample properties; consistency describes a sequence as \(n\) grows. Among unbiased estimators, a smaller variance means greater efficiency in the comparison used by this chapter. MSE provides an additional criterion when bias is allowed.
Derivation of MSE. Decompose \(\hat{\theta } -\theta =(\hat{\theta } -\mathbb{E}[\hat{\theta } ])+(\mathbb{E}[\hat{\theta } ]-\theta )\). The cross term has expectation zero. A biased estimator may have lower MSE if the variance reduction outweighs squared bias.
| Estimator of \(\mu\) under i.i.d. finite-variance sampling | Bias | Variance | Consistent? |
|---|---|---|---|
| \(Y_{1}\) | 0 | \(\sigma ^{2}\) | Generally no |
| \(\bar{Y}\) | 0 | \(\sigma ^{2}/n\) | Yes |
| \(\bar{Y} +1/n\) | \(1/n\) | \(\sigma ^{2}/n\) | Yes |
| Alternating weights \(.5/n, 1.5/n (\)even \(n\)) | 0 | \(1.25\sigma ^{2}/n\) | Yes |
2. Why the sample mean is BLUE
Consider linear estimators \(\hat{\mu } =\sum w_{i}Y_{i}\) with fixed, nonrandom weights. Unbiasedness for all \(\mu\) requires \(\sum w_{i}=1\). Under i.i.d. sampling with positive finite variance:
Therefore \(\operatorname{Var}(\hat{\mu } )\ge \sigma ^{2}/n\), with equality precisely at \(w_{i}=1/n\). Thus \(\bar{Y}\) is the Best Linear Unbiased Estimator within this class. “Best” is qualified by the class, the criterion, and the sampling assumptions. It does not compare every nonlinear or biased estimator under every loss function.
3. Why the sample mean is the least-squares constant
Given observations, minimize \(Q(m)=\sum (Y_{i}-m)^{2}\) over m.
The algebraic proof is even more revealing:
Expand \(Y_{i}-m=(Y_{i}-\bar{Y} )+(\bar{Y} -m)\); the cross term vanishes because \(\sum (Y_{i}-\bar{Y} )=0\). The second term is minimized at \(m=\bar{Y}\). This is an algebraic fact for any data, without i.i.d. or normality. Its statistical performance as an estimate of a population mean requires separate assumptions.
4. More observations do not repair selection bias
If observations come from a selected population with mean \(\mu _{S}\) rather than the target mean \(\mu\), averaging may converge to \(\mu _{S}\). Sampling unemployed people in parks during working hours illustrates this mismatch. A small standard error can coexist with a large error relative to the desired population.
The polling box in SW §3.1 emphasizes representation and response behavior. Random sampling concerns whose outcomes enter the dataset; random assignment concerns who receives a treatment. These address different inferential problems.
5. Nonlinear transformations preserve some properties
\(\mathbb{E}[\bar{Y} ^{2}]=\mu ^{2}+\sigma ^{2}/n\), so \(\bar{Y} ^{2}\) is generally biased for \(\mu ^{2}\). Yet \(\bar{Y} ^{2} \xrightarrow{p} \mu ^{2}\) by continuity of squaring and the LLN. Unbiasedness does not automatically survive a nonlinear transformation; consistency often does when continuity holds.
Check your understanding
Question. Does least-squares optimality of \(\bar{Y}\) prove that a convenience sample estimates the national mean without bias?
Solution. No. The minimization identity holds even for badly selected data. It does not establish representativeness.
Sources: SW §3.1, pp. 112–117; Appendix 3.2, p. 153; Exercises 3.11 and 3.19; Aden notes p. 16. MSE is a supporting extension.