The pearls of AP Statistics 28

Sep 16

The pearls of AP Statistics 28

category: Agresti & Franklin, Agresti and Franklin, AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2, The College Board 5 Comments

From independence of events to independence of random variables

One way to avoid complex Math is by showing the students simplified, plausible derivations which create appearance of rigor and provide enough ground for intuition. This is what I try to do here.

Independence of random variables

Let $X,Y$ be two random variables. Suppose $X$ takes values $x_1,x_2$ with probabilities $P(X=x_i)=p_i$ . Similarly, $Y$ takes values $y_1,y_2$ with probabilities $P(Y=y_i)=q_i$ . Now we want to consider a pair $(X,Y)$ . The pair can take values $(x_i,y_j)$ where $i,j$ take values $1,2$ . These are joint events with probabilities denoted $P(X=x_i,Y=y_j)=p_{i,j}$ .

Definition. $X,Y$ are called independent if for all $i,j$ one has

(1) $p_{i,j}=p_iq_j$ .

Thus, in case of two-valued variables, their independence means independence of 4 events. Independence of variables is a more complex condition than independence of events.

Properties of independent variables

Property 1. For independent variables, we have $EXY=EXEY$ (multiplicativity). Indeed, by definition of the expected value and equation (1)

$EXY=x_1y_1p_{1,1}+x_1y_2p_{1,2}+x_2y_1p_{2,1}+x_2y_2p_{2,2}$

$=x_1y_1p_1q_1+x_1y_2p_1q_2+x_2y_1p_2q_1+x_2y_2p_2q_2$

$=(x_1p_1+x_2p_2)(y_1q_1+y_2q_2)=EXY$ .

Remark. This proof is a good exercise to check how well students understand the definitions of the product $XY$ and of the expectation operator. Note also that multiplicativity holds only under independence, unlike linearity $E(aX+bY)=aEX+bEY$ , which is always true.

Property 2. Independent variables are uncorrelated: $Cov(X,Y)=0$ . This follows immediately from multiplicativity and the shortcut for covariance:

(2) $Cov(X,Y)=E(XY)-(EX)(EY)=0.$

Remark. Independence is stronger than uncorrelatedness: variables can be uncorrelated but not independent.

Property 3. For independent variables, variance is additive: $Var(X+Y)=Var(X)+Var(Y).$ This easily follows from the general formula for $Var(X+Y)$ and equation (2):

$Var(X+Y)=Var(X)+2Cov(X,Y)+Var(Y)=Var(X)+Var(Y).$

Property 4. Independence is such a strong property that it is preserved under nonlinear transformations. This means the following. Take two deterministic functions $f,g$ ; apply one to $X$ and the other to $Y$ . The resulting random variables $f(X),g(Y)$ will be independent. Instead of the proof, I provide an application. If $z_1,z_2$ are two independent standard normals, then $z^2_1,z^2_2$ are two independent chi-square variables with 1 degree of freedom.

Remark. Normality is preserved only under linear transformations.

This post is an antithesis of the following definition from (Agresti and Franklin, p.540): Two categorical variables are independent if the population conditional distributions for one of them are identical at each category of the other. The variables are dependent (or associated) if the conditional distributions are not identical.

Raising the bar

Ins and outs of quantitative courses of University of London