Wikipedia says: The magnitude of the covariance is not easy to interpret. I add: We keep the covariance around mainly for its algebraic properties. It deserves studying because it appears in two important formulas: correlation coefficient and slope estimator in simple regression (see derivation, simplified derivation and proof of unbiasedness).

**Definition**. For two random variables their **covariance** is defined by

(it's the mean value of the product of the deviations of two variables from their respective means).

### Properties of covariance

**Property 1**. **Linearity**. Covariance is linear in the first argument when the second argument is fixed: for any random variables and numbers one has

(1)

**Proof**. We start by writing out the left side of Equation (1):

(using linearity of means)

(collecting similar terms)

(distributing )

(using linearity of means)

**Exercise**. Covariance is also linear in the second argument when the first argument is fixed. Write out and prove this property. You can notice the importance of using parentheses and brackets.

**Property 2**. **Shortcut for covariance**: .

**Proof**.

(multiplying out)

( are constants; use linearity)

**Definition**. Random variables are called **uncorrelated** if .

Uncorrelatedness is close to independence, so the intuition is the same: one variable does not influence the other. You can also say that there is no statistical relationship between uncorrelated variables. The mathematical side is not the same: uncorrelatedness is a more general property than independence.

**Property 3**. **Independent variables are uncorrelated**: if are independent, then .

**Proof**. By the shortcut for covariance and multiplicativity of means for independent variables we have .

**Property 4**. **Correlation with a constant**. Any random variable is uncorrelated with any constant:

**Property 5**. **Symmetry**. Covariance is a **symmetric function of its arguments**: . This is obvious.

**Property 6**. **Relationship between covariance and variance**:

.

## Leave a Reply

You must be logged in to post a comment.