Sep 16

The pearls of AP Statistics 28

From independence of events to independence of random variables

One way to avoid complex Math is by showing the students simplified, plausible derivations which create appearance of rigor and provide enough ground for intuition. This is what I try to do here.

Independence of random variables

Let X,Y be two random variables. Suppose X takes values x_1,x_2 with probabilities P(X=x_i)=p_i. Similarly, Y takes values y_1,y_2 with probabilities P(Y=y_i)=q_i. Now we want to consider a pair (X,Y). The pair can take values (x_i,y_j) where i,j take values 1,2. These are joint events with probabilities denoted P(X=x_i,Y=y_j)=p_{i,j}.

DefinitionX,Y are called independent if for all i,j one has

(1) p_{i,j}=p_iq_j.

Thus, in case of two-valued variables, their independence means independence of 4 events. Independence of variables is a more complex condition than independence of events.

Properties of independent variables

Property 1. For independent variables, we have EXY=EXEY (multiplicativity). Indeed, by definition of the expected value and equation (1)

EXY=x_1y_1p_{1,1}+x_1y_2p_{1,2}+x_2y_1p_{2,1}+x_2y_2p_{2,2} =x_1y_1p_1q_1+x_1y_2p_1q_2+x_2y_1p_2q_1+x_2y_2p_2q_2


Remark. This proof is a good exercise to check how well students understand the definitions of the product XY and of the expectation operator. Note also that multiplicativity holds only under independence, unlike linearity E(aX+bY)=aEX+bEY, which is always true.

Property 2. Independent variables are uncorrelated: Cov(X,Y)=0. This follows immediately from multiplicativity and the shortcut for covariance:

(2) Cov(X,Y)=E(XY)-(EX)(EY)=0.

Remark. Independence is stronger than uncorrelatedness: variables can be uncorrelated but not independent.

Property 3. For independent variables, variance is additive: Var(X+Y)=Var(X)+Var(Y). This easily follows from the general formula for Var(X+Y) and equation (2):


Property 4. Independence is such a strong property that it is preserved under nonlinear transformations. This means the following. Take two deterministic functions f,g; apply one to X and the other to Y. The resulting random variables f(X),g(Y) will be independent. Instead of the proof, I provide an application. If z_1,z_2 are two independent standard normals, then z^2_1,z^2_2 are two independent chi-square variables with 1 degree of freedom.

Remark. Normality is preserved only under linear transformations.

This post is an antithesis of the following definition from (Agresti and Franklin, p.540): Two categorical variables are independent if the population conditional distributions for one of them are identical at each category of the other. The variables are dependent (or associated) if the conditional distributions are not identical.

5 Responses for "The pearls of AP Statistics 28"

  1. […] Definition of independent discrete random variables easily modifies for the continuous case. Let be two continuous random variables with densities , respectively. We say that these variables are independent if the density of the pair  is a product of individual densities: […]

  2. […] 3. For independent variables, we have  (multiplicativity), which has important implications on its […]

  3. […] is close to independence, so the intuition is the same: one variable does not influence the other. You can also say that […]

  4. […] Similarly, the discount  takes values  with probabilities , . The joint events have joint probabilities denoted . The profit in the event  is denoted . This information is summarized in Table […]

  5. […] 2. Assume that observations are independent. Then the joint density is a product of own densities: . Since the observations are fixed, the joint density is a function of just […]

Leave a Reply

You must be logged in to post a comment.