Raising the bar

Chapter 6. Continuous Random Variables and Probability Distributions

Unit 6.2. Density

Recall the short definition of a discrete random variable: values plus probabilities. Regarding the probabilities, we have established two properties: they are percentages and they satisfy the completeness axiom. We want an equivalent of these notions and properties in the continuous case.

Density of a continuous random variable. Let $latex X$ be a continuous random variable and let $latex {F_X}$ be its distribution function. If there exists a function $latex {p_X}$ such that

$latex {F_X}(x) = \int\limits_{ - \infty }^x {{p_X}(t)dt} \textrm{ for all } -\infty < x < \infty, \qquad (\textrm{Eq. 6.3})$

then we say that the function $latex {p_X}$ is the density of $latex X$.

Comments. (1) Please follow my practice of attaching subscripts to emphasize that $latex {F_X}$ and $latex {p_X}$ pertain to $latex X$.

(2) The definition appeals to existence of a function $latex {p_X}$ satisfying Eq. (6.3). All continuous random variables that do not satisfy this existence requirement are left out in this course.

Property 1. From Eq. (6.3) we can obtain a more general equation (an interval formula in terms of densities):

$latex P(a < X \le b) = \int\limits_a^b {{p_X}(t)dt.} \qquad (\textrm{Eq. 6.4})$

Proof. Use the interval formula Eq. (5.16) (see Distribution function) and additivity of integrals to get

$latex P(a < X \le b) = {F_X}(b) - {F_X}(a) = \int\limits_{ - \infty }^b {{p_X}(t)dt} - \int\limits_{ - \infty }^a {{p_X}(t)dt} = \int\limits_a^b {{p_X}(t)dt.}$

Property 2. The probability of $latex X$ taking any specific value is zero because by Eq. (6.2)

$latex P(X = a) = P(X \le a) - P(X < a) = \int\limits_a^a {{p_X}(t)dt} = 0.$

For this reason $latex P(X \le x)$ is the same as $latex P(X < x)$ and $latex P(a < X < b)$ is the same as $latex P(a \le X \le b)$.

Property 3. $latex {p_X}(t) \ge 0$ for all $latex t$.

Proof. If the density is negative at some point $latex {x_0}$, then it is negative in some neighborhood of that point. We can take $latex a$ and $latex b$ close to that point and then by the interval formula Eq. (6.4) $latex P(a < X \le b)$ will be negative, which is impossible (illustrate this argument with a graph!)

Property 4. Any density satisfies the completeness axiom in the form

$latex \int\limits_{ - \infty }^{ + \infty } {{p_X}(t)dt} = 1$

(the total area under the graph is 1).

Proof. This is because $latex { -\infty < X < +\infty }$ is a sure event and $latex {F_X}(+\infty) = 1$.

Counter-example to the cow-and-goat principle. (i) In the discrete case the variable takes its values with positive probabilities $latex P(X = {x_i}) = {p_i}$. In the continuous case $latex P(X = x) = 0$ for any $latex x$.

(ii) In the discrete case the numbers $latex {p_i}$ are probabilities (percentages). In the continuous case the values of the density $latex {p_X}(t)$ mean nothing and can be larger than 1. It's the area $latex \int\limits_a^b {{p_X}(t)dt}$ that means probability!

Exercise 6.1. Take two related continuous random variables and draw what you think would be their densities. For example, you can take:

1. distributions of income in a wealthy neighborhood and in a poor neighborhood,
2. distributions of temperature in winter and summer in a given geographic location;
3. distributions of electricity consumption in two different locations at the same time of the year.

Don't forget that the total density is 1!

Exercise 6.2.

(a) Suppose $latex Y$ is a linear transformation of $latex X$, $latex Y = aX + b$, where $latex a > 0$. What is the relationship between the distribution functions of $latex Y$ and $latex X$?

(b) Further assume that $latex X$ has a density $latex {p_X}$. What is the density of $latex Y$?

Solution. (a) Directly from definitions

$latex {F_Y}(y) = P(Y \le y) = P(aX + b \le y) = P\left( X \le \frac{{y - b}}{a} \right) = {F_X}\left( \frac{{y - b}}{a} \right).$

(b) The equation just obtained with the help of densities rewrites as follows:

$latex \int_{ - \infty }^y {{p_Y}(t)dt} = {F_Y}(y) = {F_X}\left( \frac{{y - b}}{a} \right) = \int_{ - \infty }^{(y - b)/a} {{p_X}(t)dt}.$

Now we need two rules from calculus:

1. if $latex F(y) = \int_{ - \infty }^y {p(t)dt}$, then $latex \frac{{dF}}{{dy}} = p(y)$ (derivative with respect to upper limit of integration). This rule is enough to differentiate the left side.
2. For the right side we need also the chain rule for finding a derivative of a composite function $latex f(x) = g(h(x)).$ Here $latex g(h)$ can be called an outer function and $latex h(x)$ an inner function. The chain rule says that the derivative of $latex f$ can be found as a product of derivatives of the outer and inner functions: $latex f'(x) = g'(h(x))h'(x).$ In the expression $latex g'(h(x))$ one has to first differentiate $latex g$ and then plug $latex h(x)$ in.

Differentiation of the above equation leads to

$latex {p_Y}(y) = \frac{d}{{dy}}\int_{ - \infty }^y {{p_Y}(t)dt} = \frac{d}{{dy}}\int_{ - \infty }^{(y - b)/a} {{p_X}(t)dt} = \frac{1}{a}{p_X}\left( \frac{{y - b}}{a} \right).$

Previous: Properties of integrals | Next: Mean value of a continuous random variable

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.