Sep 16

The pearls of AP Statistics 28

From independence of events to independence of random variables

One way to avoid complex Math is by showing the students simplified, plausible derivations which create appearance of rigor and provide enough ground for intuition. This is what I try to do here.

Independence of random variables

Let X,Y be two random variables. Suppose X takes values x_1,x_2 with probabilities P(X=x_i)=p_i. Similarly, Y takes values y_1,y_2 with probabilities P(Y=y_i)=q_i. Now we want to consider a pair (X,Y). The pair can take values (x_i,y_j) where i,j take values 1,2. These are joint events with probabilities denoted P(X=x_i,Y=y_j)=p_{i,j}.

DefinitionX,Y are called independent if for all i,j one has

(1) p_{i,j}=p_iq_j.

Thus, in case of two-valued variables, their independence means independence of 4 events. Independence of variables is a more complex condition than independence of events.

Properties of independent variables

Property 1. For independent variables, we have EXY=EXEY (multiplicativity). Indeed, by definition of the expected value and equation (1)




Remark. This proof is a good exercise to check how well students understand the definitions of the product XY and of the expectation operator. Note also that multiplicativity holds only under independence, unlike linearity E(aX+bY)=aEX+bEY, which is always true.

Property 2. Independent variables are uncorrelated: Cov(X,Y)=0. This follows immediately from multiplicativity and the shortcut for covariance:

(2) Cov(X,Y)=E(XY)-(EX)(EY)=0.

Remark. Independence is stronger than uncorrelatedness: variables can be uncorrelated but not independent.

Property 3. For independent variables, variance is additive: Var(X+Y)=Var(X)+Var(Y). This easily follows from the general formula for Var(X+Y) and equation (2):


Property 4. Independence is such a strong property that it is preserved under nonlinear transformations. This means the following. Take two deterministic functions f,g; apply one to X and the other to Y. The resulting random variables f(X),g(Y) will be independent. Instead of the proof, I provide an application. If z_1,z_2 are two independent standard normals, then z^2_1,z^2_2 are two independent chi-square variables with 1 degree of freedom.

Remark. Normality is preserved only under linear transformations.

This post is an antithesis of the following definition from (Agresti and Franklin, p.540): Two categorical variables are independent if the population conditional distributions for one of them are identical at each category of the other. The variables are dependent (or associated) if the conditional distributions are not identical.

Leave a Reply

You must be logged in to post a comment.