Raising the bar

Distribution and density functions of a linear transformation - two short derivations to read during breakfast.

Distribution function of a linear transformation

Let $latex X$ be a random variable and let $latex Y=aX+b$ be its linear transformation (here $latex a,b$ are some real numbers and $latex a\ne0$, otherwise $latex Y$ is not random). If the distribution function $latex F_X$ is known, what will be the distribution function of $latex Y$?

The answer is obtained in one line if you know the definition of the distribution function:

(1) $latex F_Y(y)=P(Y\le y)=P(aX+b\le y)=P(X\le\frac{y-b}{a})=F_X(\frac{y-b}{a})$.

For the inequalities $latex aX+b\le y$ and $latex X\le\frac{y-b}{a}$ to be equivalent, we have to assume that $latex a>0$ (for applications this is enough). The case $latex a<0$ is left as an exercise.

Density function of a linear transformation

As above, $latex Y$ is a linear transformation of $latex X$. Suppose they have densities $latex p_X,p_Y$. What is the relationship between the densities?

Recall formula (1) that links distribution and density functions. Equation (1) in terms of densities becomes

$latex \int_{-\infty}^yp_Y(t)dt=\int_{-\infty}^{\frac{y-b}{a}}p_X(t)dt$.

Let's differentiate this equation. The Newton-Leibnitz formula applied to the integral on the left gives $latex p_Y$ evaluated at $latex y$. On the right, additionally, we have to use the chain rule. The result is

(2) $latex p_Y(y)=p_X(\frac{y-b}{a})\frac{d}{dy}\frac{y-b}{a}=\frac{1}{a}p_X(\frac{y-b}{a})$.

Equation (2) will be used to derive ML estimators for the linear model.