Dependence & Covariance Algebra

Independence, mean independence, and zero correlation are different restrictions. Covariance algebra tells us when averaging reduces uncertainty and when common shocks survive.

1. A hierarchy of dependence restrictions

Independence: the joint distribution factors into the marginals. Equivalently, knowing \(X\) does not change the distribution of \(Y\). Mean independence of \(Y\) from \(X\): \(\mathbb{E}[Y\mid X]=\mathbb{E}[Y]\). Zero covariance: \(\mathbb{E}[(X-\mu _{X})(Y-\mu _{Y})]=0\). With the required moments:

\[\begin{gathered}X\perp\!\!\!\perp Y\\ \Downarrow\\ \mathbb E[Y\mid X]=\mathbb E[Y]\\ \Downarrow\\ \operatorname{Cov}(X,Y)=0.\end{gathered}\]

Mean independence is directional. Independence is symmetric. Neither reverse implication is generally valid.

Counterexample to zero covariance \(\implies\) mean independence. Let \(X\) and \(Z\) be independent \(\mathcal{N}(0,1)\) and \(Y=X^{2}+Z\). Then \(\mathbb{E}[Y\mid X]=X^{2}, \mathbb{E}[Y]=1\), and \(\mathbb{E}[\)XY\(]=\mathbb{E}[X^{3}]+\mathbb{E}[\)XZ]=0. Thus \(\operatorname{Cov}(X,Y)=0\) although \(X\) predicts the conditional mean of \(Y\).

Counterexample to mean independence \(\implies\) independence. Let \(X\) take values 1 and 2 with equal probability and let \(\varepsilon\) independently take \(\pm 1\) with equal probability. Set \(Y=X\varepsilon . \mathbb{E}[Y\mid X]=0\), but |Y| reveals \(X\). The conditional spread depends on \(X\).

2. Covariance and correlation

\[\begin{aligned}\sigma_{XY}&=\operatorname{Cov}(X,Y)\\&=\mathbb E[(X-\mu_X)(Y-\mu_Y)]\\&=\mathbb E[XY]-\mu_X\mu_Y,\\ \rho_{XY}&=\frac{\sigma_{XY}}{\sigma_X\sigma_Y},\qquad -1\le\rho_{XY}\le1.\end{aligned}\]

Correlation requires finite, positive variances. Covariance has product units; correlation is unitless. Correlation measures linear association, not slope or all forms of dependence. Correlation \(\pm 1\) means an exact affine relationship almost surely, with positive or negative slope.

Why mean independence implies zero covariance. \(\mathbb{E}[\)XY\(]=\mathbb{E}[X \mathbb{E}[Y\mid X]]=\mathbb{E}[X]\mathbb{E}[Y\)], so subtracting \(\mu _{X}\mu _{Y}\) gives zero. This uses iterated expectations, not an assumption of full independence.

3. Algebra for weighted sums

\[\begin{aligned}\operatorname{Cov}(a+bX+cV,Y)&=b\operatorname{Cov}(X,Y)+c\operatorname{Cov}(V,Y),\\ \operatorname{Var}(aX+bY)&=a^2\sigma_X^2+b^2\sigma_Y^2+2ab\sigma_{XY},\\ \operatorname{Var}(X-Y)&=\sigma_X^2+\sigma_Y^2-2\sigma_{XY},\\ \operatorname{Var}\!\left(\sum_iw_iY_i\right)&=\sum_iw_i^2\operatorname{Var}(Y_i)\\&\quad+2\sum_{i<j}w_iw_j\operatorname{Cov}(Y_i,Y_j).\end{aligned}\]

Derivation. Center each variable, expand the square of the weighted sum, and take expectations term by term. Independence is needed only if it is used to set cross-covariances to zero. For a difference, the coefficients are 1 and \(-1\), which explains the minus sign on the covariance term.

Correlation inequality, Appendix 2.1. For \(\sigma _{X}^{2}>0\), nonnegativity of \(\operatorname{Var}(Y-\)aX) at \(a=\sigma _{XY}/\sigma _{X}^{2}\) gives \(0\le \sigma _{Y}^{2}-\sigma _{XY}^{2}/\sigma _{X}^{2}\). Hence \(\sigma _{XY}^{2}\le \sigma _{X}^{2}\sigma _{Y}^{2}\) and \(|\rho |\le 1\). This is the covariance form of the Cauchy–Schwarz inequality.

4. Diversification and dependence

For an equally weighted average of \(n\) variables with common mean \(\mu\), common variance \(\sigma ^{2}\), and common pairwise correlation \(\rho\):

\[\mathbb E[\bar Y]=\mu,\qquad \operatorname{Var}(\bar Y)=\frac{\sigma^2}{n}+\frac{n-1}{n}\rho\sigma^2.\]

There are \(n\) variance terms and \(n(n-1)\) ordered covariance terms in the expanded square. If \(\rho =0\), the variance falls as \(1/n\). For fixed positive \(\rho\), it approaches \(\rho \sigma ^{2}\). The common-correlation covariance matrix must be valid; in particular \(-1/(n-1)\le \rho \le 1. A\) fixed negative \(\rho\) cannot persist for arbitrarily large \(n\).

Two-asset example. \(R=\)wR\(_{s}+(1-w)R_{b}\), with means .08 and .05, standard deviations .07 and .04, and correlation .25. At \(w=.5, \mathbb{E}[R]=.065\) and \(\operatorname{Var}(R)=.001975\), so \(\operatorname{SD}(R)\approx .04444\). The minimum-variance weight without additional constraints is \(w*=(\sigma _{b}^{2}-\sigma _{sb})/(\sigma _{s}^{2}+\sigma _{b}^{2}-2\sigma _{sb})\approx .17647\). This is a mathematical example using textbook inputs.

Check your understanding

Question. If two measurements have positive covariance, does subtracting one from the other increase or decrease the variance relative to independent measurements with the same marginal variances?

Solution. It decreases it by \(2Cov(X,Y)\). This is why paired comparisons can be more precise.

Sources: SW §2.3, pp. 77–81; diversification box p. 92; Exercises 2.22–2.23, 2.26; Appendix 2.1, pp. 109–110. Counterexample for mean independence and validity bound are supporting clarifications.