Raising the bar

Chapter 6. Continuous Random Variables and Probability Distributions

Unit 6.9. Exponential distribution and link to Poisson

Motivating example. You are watching phone calls made by Verizon customers. At some point you turn on your stopwatch, to see how much time will pass till the first call. This waiting time is the variable of interest. Since time is continuous and the time of arrival of the first call is unpredictable, this is a continuous random variable. Moreover, because the time of arrival of the first call cannot be negative, its density function will be zero on the left half-axis. The density can take many different functional forms. Empirical evidence shows that the following specification is adequate.

Exponential distribution. By definition, the density of the exponential distribution is

$latex p(t) = 0,\ \textrm{if } t \le 0,\ p(t) = \mu {e^{ - \mu t}},\ \textrm{if } t > 0. \qquad (\textrm{Eq. 6.23})$

where $latex \mu$ is a positive parameter.

Exercise 6.6. Check that this is really a density. Find the mean and variance of this variable.

Solution. For clarity, it is useful to separate facts from calculus and those from statistics.

Fact 1. Recall l'Hôpital's rule: let $latex {\lim _{x \to \infty }}f(x) = {\lim _{x \to \infty }}g(x) = \infty$, so that the limit $latex {\lim _{x \to \infty }}\frac{{f(x)}}{{g(x)}}$ cannot be found directly by plugging $latex x = \infty$. Suppose that the limit $latex {\lim _{x \to \infty }}\frac{{f'(x)}}{{g'(x)}}$ exists. Then $latex {\lim _{x \to \infty }}\frac{{f(x)}}{{g(x)}} = {\lim _{x \to \infty }}\frac{{f'(x)}}{{g'(x)}}$.

This rule allows us to find the limit

$latex {\lim _{t \to \infty }}t{e^{ - \mu t}} = {\lim _{t \to \infty }}\frac{t}{{{e^{\mu t}}}} = {\lim _{t \to \infty }}\frac{{t'}}{{({e^{\mu t}})'}} = {\lim _{t \to \infty }}\frac{1}{{\mu {e^{\mu t}}}} = 0. \qquad (\textrm{Eq. 6.24})$

Similarly, applying l'Hôpital's rule twice,

$latex {\lim _{t \to \infty }}{t^2}{e^{ - \mu t}} = 0. \qquad (\textrm{Eq. 6.25})$

Fact 2. Integration by parts: If $latex f(x),g(x)$ have primitives $latex F(x),G(x)$, respectively, then

$latex \int_a^b {F(x)g(x)dx} = \left. {F(x)G(x)} \right|_a^b - \int_a^b {f(x)G(x)dx.}$

Now we are able to calculate three integrals.

(1) The primitive of $latex {e^{ - \mu t}}$ is $latex -\frac{1}{\mu }{e^{ - \mu t}}$, so

$latex \int_0^\infty {{e^{ - \mu t}}} dt = \left. { - \frac{1}{\mu }{e^{ - \mu t}}} \right|_0^\infty = \frac{1}{\mu }. \qquad (\textrm{Eq. 6.26})$

(2) The primitive of 1 is $latex t$ and the primitive of $latex {e^{ - \mu t}}$ is $latex -\frac{1}{\mu }{e^{ - \mu t}}$, so Eq. (6.24) and Eq. (6.26) imply

$latex \int_0^\infty {t{e^{ - \mu t}}dt} = \left. {t\left( { - \frac{1}{\mu }{e^{ - \mu t}}} \right)} \right|_0^\infty + \frac{1}{\mu }\int_0^\infty {{e^{ - \mu t}}dt} = \frac{1}{{{\mu ^2}}}. \qquad (\textrm{Eq. 6.27})$

(3) Further, since the primitive of $latex 2t$ is $latex {t^2}$ and the primitive of $latex {e^{ - \mu t}}$ is $latex -\frac{1}{\mu }{e^{ - \mu t}}$, we have by equations (6.25) and (6.27)

$latex \int_0^\infty {{t^2}{e^{ - \mu t}}} dt = \left. {{t^2}\left( { - \frac{1}{\mu }{e^{ - \mu t}}} \right)} \right|_0^\infty + \int_0^\infty {2t\left( {\frac{1}{\mu }{e^{ - \mu t}}} \right)} dt = \frac{2}{\mu }\int_0^\infty {t{e^{ - \mu t}}} dt = \frac{2}{{{\mu ^3}}}.$

Now it is easy to finish the solution

$latex \int_{ - \infty }^\infty {p(t)dt} = 1,$

$latex EX = \int_{ - \infty }^\infty {tp(t)dt} = \mu \int_0^\infty {t{e^{ - \mu t}}dt} = \frac{1}{\mu },$

$latex E{X^2} = \int_{ - \infty }^\infty {{t^2}p(t)dt} = \mu \int_0^\infty {{t^2}{e^{ - \mu t}}dt} = \frac{2}{{{\mu ^2}}},$

$latex V(X) = E{X^2} - {(EX)^2} = \frac{1}{{{\mu ^2}}}.$

Interpretation of the parameter $latex \mu$ as the inverse of $latex EX$ is not very appealing. Below we'll see that there is a different interpretation. We turn to the link between the exponential and Poisson distributions.

Exercise 6.7. You continue watching calls by Verizon customers but ask a different question. Suppose phone calls are made at the rate $latex \mu$ per unit of time, independently of each other and following the Poisson distribution. Denote by $latex Y$ the waiting time until the first call. What is the distribution of $latex Y?$

Solution. We know everything about $latex Y$ if we know its distribution function. So we start with $latex {F_Y}(y)$. By the complement rule

$latex {F_Y}(y) = 1 - P(Y > y)$

$latex = 1 - P(\textrm{waiting time exceeds } y)$

$latex = 1 - P(\textrm{no calls are made in the period } (0,y]). \qquad (\textrm{Eq. 6.28})$

This is the place to use the Poisson distribution. If the call rate is $latex \mu$ per unit of time, then, logically, it is $latex \mu y$ per time period $latex (0,y]$ (this is also true for fractional $latex y$). By Eq. (5.18) (see Poisson distribution) $latex P(\textrm{no calls are made in the period } (0,y]) = {e^{ - \mu y}}$ and then from Eq. (6.28) we have $latex {F_Y}(y) = 1 - {e^{ - \mu y}}$. From this equality we need to extract the density of $latex Y$. This is done with the help of the derivative with respect to the upper limit of integration, see the solution to Exercise 6.2 (in Density).

We have proved a general fact: if events occur independently according to the Poisson law at rate $latex \mu$ per unit of time, then the waiting time follows an exponential distribution with the same parameter.

Previous: Joint distribution and independence | Next: Questions for repetition (Chapter 6)

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.