Raising the bar

The magic of the distribution function

Let $latex X$ be a random variable. The function $latex F_{X}\left( x\right) =P\left(
X\leq x\right) ,$ where $latex x$ runs over real numbers, is called a distribution function of $latex X.$ In statistics, many formulas are derived with the help of $latex F_{X}\left( x\right) .$ The motivation and properties are given here.

Oftentimes, working with the distribution function is an intermediate step to obtain a density $latex f_{X}$ using the link

$latex F_{X}\left( x\right) =\int_{-\infty }^{x}f_{X}\left( t\right) dt.$

A series of exercises below show just how useful the distribution function is.

Exercise 1. Let $latex Y$ be a linear transformation of $latex X,$ that is, $latex Y=\sigma X+\mu ,$ where $latex \sigma >0$ and $latex \mu \in R.$ Find the link between $latex F_{X}$ and $latex F_{Y}.$ Find the link between $latex f_{X}$ and $latex f_{Y}.$

The solution is here.

The more general case of a nonlinear transformation can also be handled:

Exercise 2. Let $latex Y=g\left( X\right) $ where $latex g$ is a deterministic function. Suppose that $latex g$ is strictly monotone and differentiable. Then $latex g^{-1}$ exists. Find the link between $latex F_{X}$ and $latex F_{Y}.$ Find the link between $latex f_{X}$ and $latex f_{Y}.$

Solution. The result differs depending on whether $latex g$ is increasing or decreasing. Let's assume the latter, so that $latex x_{1}\leq x_{2}$ is equivalent to $latex g\left( x_{1}\right) \geq g\left( x_{2}\right) .$ Also for simplicity suppose that $latex P\left( X=c\right) =0$ for any $latex c\in R.$ Then

$latex F_{Y}\left( y\right) =P\left( g\left( X\right) \leq y\right) =P\left( X\geq g^{-1}\left( y\right) \right) =1-P\left( X\leq g^{-1}\left( y\right) \right)=1-F_{X}\left( g^{-1}\left( y\right) \right) .$

Differentiation of this equation produces

$latex f_{Y}\left( y\right)=-f_{X}\left( g^{-1}\left( y\right) \right) \left( g^{-1}\left( y\right)
\right) ^{\prime }=f_{X}\left( g^{-1}\left( y\right) \right) \left\vert\left( g^{-1}\left( y\right) \right) ^{\prime }\right\vert $

(the derivative of $latex g^{-1}$ is negative).

For an example when $latex g$ is not invertible see the post about the chi-squared distribution.

Exercise 3. Suppose $latex T=X/Y$ where $latex X$ and $latex Y$ are independent, have densities $latex f_{X},f_{Y}$ and $latex Y>0.$ What are the distribution function and density of $latex T?$

Solution. By independence the joint density $latex f_{X,Y}$ equals $latex f_{X}f_{Y},$ so

$latex F_{T}\left( t\right) =P\left( T\leq t\right) =P\left( X\leq tY\right) = \underset{x\leq ty}{\int \int }f_{X}\left( x\right) f_{Y}\left( y\right) dxdy $

(converting a double integral to an iterated integral and remembering that $latex f_{Y}$ is zero on the left half-axis)

$latex =\int_{0}^{\infty }\left( \int_{-\infty }^{ty}f_{X}\left( x\right) dx\right) f_{Y}\left( y\right)dy=\int_{0}^{\infty }F_{X}\left( ty\right) f_{Y}\left( y\right) dy.$

Now by the Leibniz integral rule

(1) $latex f_{T}\left( t\right) =\int_{0}^{\infty }f_{X}\left( ty\right) yf_{Y}\left( y\right) dy.$

A different method is indicated in Activity 4.11, p.207 of J.Abdey, Guide ST2133.

Exercise 4. Let $latex X,Y$ be two independent random variables with densities $latex f_{X},f_{Y}$. Find $latex F_{X+Y}$ and $latex f_{X+Y}.$

See this post.

Exercise 5. Let $latex X,Y$ be two independent random variables. Find $latex F_{\max \left\{ X_{1},X_{2}\right\}}$ and $latex F_{\min \left\{ X_{1},X_{2}\right\}}$.

Solution. The inequality $latex \max \left\{ X_1,X_2\right\} \leq x$ holds if and only if both $latex X_{1}\leq x$ and $latex X_{2}\leq x$ hold. This means that the event $latex \left\{\max \left\{ X_{1},X_{2}\right\} \leq x\right\} $ coincides with the event $latex \left\{ X_{1}\leq x\right\} \cap \left\{ X_2\leq x\right\}$. It follows by independence that

(2) $latex F_{\max \left\{ X_{1},X_{2}\right\} }\left( x\right) =P\left( \max \left\{ X_{1},X_{2}\right\} \leq x\right) =P\left( \left\{ X_{1}\leq x\right\} \cap \left\{ X_{2}\leq x\right\} \right) $

$latex =P(X_{1}\leq x)P\left(X_{2}\leq x\right) =F_{X_{1}}\left( x\right) F_{X_{2}}\left( x\right) .$

For $latex \min \left\{ X_{1},X_{2}\right\} $ we need one more trick, namely, pass to the complementary event by writing

$latex F_{\min \left\{ X_{1},X_{2}\right\} }\left(x\right) =P\left( \min \left\{ X_{1},X_{2}\right\} \leq x\right) =1-P\left(\min \left\{ X_{1},X_{2}\right\} >x\right) $.

Now we can use the fact that the event $latex \left\{ \min \left\{ X_{1},X_{2}\right\} >x\right\} $ coincides with the event $latex \left\{ X_{1}>x\right\} \cap \left\{ X_{2}>x\right\} .$ Hence, by independence

(3) $latex F_{\min \left\{ X_{1},X_{2}\right\} }\left( x\right) =1-P\left( \left\{X_{1}>x\right\} \cap \left\{ X_{2}>x\right\}\right) =1-P\left(X_{1}>x\right) P\left( X_{2}>x\right) $

$latex =1-\left[ 1-P\left( X_{1}\leq x\right) \right] \left[ 1-P\left( X_{2}\leq
x\right) \right] =1-\left( 1-F_{X_{1}}\left( x\right) \right) \left(1-F_{X_{2}}\left( x\right) \right) .$

Equations (2) and (3) can be differentiated to obtain the links in terms of densities.

Exercise 6. Derive the marginal rules for distribution functions and for densities.

Solution. The definition of the distribution function $latex F_{X}\left( x\right) =P\left(X\leq x\right) $ for one random variable easily  generalizes to the case of a pair of random variables: $latex F_{X,Y}\left( x,y\right) =P\left( X\leq x,Y\leq y\right) .$ Since with $latex y=\infty $ in the last definition there is no dependence on $latex Y$ (the event $latex \left\{ Y<\infty \right\} $ is a sample space for $latex Y),$ we obtain and important link

(4) $latex F_{X}\left( x\right) =F_{X,Y}\left( x,\infty \right) .$

Write this using densities:

$latex \int_{-\infty }^{x}f_{X}\left( t\right) dt=\int_{-\infty }^{x}\left(
\int_{-\infty }^{\infty }f_{X,Y}\left( x,y\right) dy\right) dx$

and differentiate to get

(5) $latex f_{X}\left( x\right) =\int_{-\infty }^{\infty }f_{X,Y}\left( x,y\right) dy.$

(1) is the marginal rule for distribution functions and (2) is the marginal
rule for densities.