Raising the bar

Chapter 7. Sampling and Sampling Distribution

Unit 7.3. Monte Carlo simulations

Here we explain how the Bernoulli, binomial and normal variables can be modeled on the computer. Understanding this topic is not required by AP Stats or LSE Stats courses, so the whole unit can be safely skipped.

Exercise 7.4 (Modeling Bernoulli). Let $latex U = U_{(0,1)}$ denote the variable that is uniformly distributed over $latex (0,1)$. For $latex 0 < p < 1$ put $latex X = 0$ if $latex U > p$, and $latex X = 1$ if $latex U \le p$ (Eq. 7.4). Prove that then $latex X$ is a Bernoulli variable and $latex P(X = 1) = p$.

Proof. $latex X$ takes only values 0 and 1. By the definition of $latex U$ we have $latex P(X = 1) = P(U \le p) = \int_0^p dt = p. \quad \blacksquare$

In Excel the function RAND() supplies the variable $latex U$. The mathematical definition Eq. (7.4) in Excel is realized by the command IF(condition,$latex v_1$,$latex v_2$). If condition is true, it gives $latex v_1$, and if it is wrong, the command gives $latex v_2$. Thus, Exercise 2.1 models a Bernoulli variable with $latex p = 1/4$.

Exercise 7.5 (Modeling binomial). How would you model the binomial variable $latex S_3$ in Excel?

Solution. With three independent variables $latex X_1, X_2, X_3$ defined by Eq. (7.4) put $latex S_3 = X_1 + X_2 + X_3$. When you put the function RAND() in different cells, the resulting variables will be independent. Therefore Exercise 2.2 models the binomial for which $latex ES_3 = 3p = 3/4$. $latex \blacksquare$

Before we move on to modeling continuous random variables we need to review the notion of an inverse function.

Definition 7.6 (Inverse function as an equation solution). Let $latex f$ be a function with the domain $latex D(f)$ and range $latex R(f)$. If for any value $latex y \in R(f)$ the equation $latex f(x) = y$ has a unique solution $latex x$, we say that $latex f$ has an inverse function $latex f^{-1}$ and write the solution as $latex x = f^{-1}(y)$.

In equations try to write the knowns on the right and the unknowns (or what you define) on the left side.

Definition 7.7 (Inverse function $latex -$ working definition). $latex f$ is invertible if the equality of values $latex f(x_1) = f(x_2)$ implies the equality of arguments $latex x_1 = x_2$ (then the equation $latex f(x) = y$ cannot have more than one solution).

Example 7.2. Consider the function $latex y = x^2$ on the real line. It does not have an inverse function because for any value $latex y > 0$ there are two different solutions of the equation $latex x^2 = y$ (can you show them on the plane?). The same function on a smaller domain $latex D(f) = [0, \infty)$ has an inverse because a positive square root of a positive number is unique. Thus, a function may not be invertible globally (on the whole domain) but may be invertible locally (on a subset of the domain).

Property 1. The direct function maps (sends) the argument to the value: $latex y = f(x)$. The inverse function maps the value to the argument: $latex x = f^{-1}(y)$. A successive application of the two mappings amounts to an identical mapping: $latex f(f^{-1}(y)) = y$.

Property 2. $latex f$ is called nondecreasing if $latex x_1 \le x_2$ implies $latex f(x_1) \le f(x_2)$. (We have seen that a distribution function is nondecreasing.) Suppose $latex f$ is invertible. Then $latex f$ is nondecreasing if and only if its inverse is nondecreasing.

Proof. Let $latex f$ be nondecreasing and let us prove that its inverse is nondecreasing. Suppose the opposite: there exist $latex y_1 > y_2$ such that $latex f^{-1}(y_1) \le f^{-1}(y_2)$. Then by monotonicity of the direct function $latex y_1 = f(f^{-1}(y_1)) \le f(f^{-1}(y_2)) = y_2$ which contradicts our assumption. Hence, the assumption is wrong and the inverse is nondecreasing.

Conversely, let $latex f^{-1}$ be nondecreasing and let us prove that $latex f$ is nondecreasing. This follows from what we've proved and the fact that $latex (f^{-1})^{-1} = f$ (a function is the inverse of its inverse). $latex \blacksquare$

Condition sufficient for invertibility of a distribution function. The distribution function $latex F_X$ of a random variable $latex X$ is invertible where the density $latex p_X$ is positive.

Proof. Suppose $latex p_X$ is positive in some interval $latex [a,b]$ and let us show that the distribution function is invertible in that interval. Suppose $latex a \le x_1 < x_2 \le b$. Then by the interval formula $latex 0 < \int_{x_1}^{x_2} p_X(t) dt = F_X(x_2) - F_X(x_1)$. Thus, the values $latex F_X(x_2), F_X(x_1)$ cannot be equal and by the working definition $latex F_X^{-1}$ exists. $latex \blacksquare$

Corollary 7.1. For a normal variable its distribution function is invertible everywhere. The distribution function of the chi-square is invertible on the half-axis $latex [0, \infty)$. For $latex U_{(a,b)}$ its distribution function is invertible on $latex [a,b]$.

Exercise 7.6 (Modeling a continuous $latex X$ with invertible $latex F_X$). How would you model an arbitrary random variable $latex X$ with an invertible distribution function $latex F_X$?

Solution. Put $latex Y = F_X^{-1}(U)$ where $latex U$ is uniformly distributed on $latex (0,1)$. Since $latex F_X$ is nondecreasing, the inequality $latex F_X^{-1}(U) \le x$ is equivalent to $latex U = F_X(F_X^{-1}(U)) \le F_X(x)$. Therefore for any real $latex x$

$latex F_Y(x) = P(F_X^{-1}(U) \le x) = P(U \le F_X(x)) = \int_0^{F_X(x)} dt = F_X(x). \quad \blacksquare$

Despite being short, this proof combines several important ideas. Make sure you understand them.

In Excel the function NORMINV($latex y$,$latex \mu$,$latex \sigma$) denotes $latex F_X^{-1}(y)$ where $latex X = \sigma z + \mu$ is normal. Hence, in Exercise 2.4 we model the normal variable $latex X = 8z + 25$.

Previous: Sampling distribution of sample variance | Next: Questions for repetition (Chapter 7)

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.