ST104A - Raising the bar

Apr 23

The magic of the distribution function

category: Advanced Statistics ST2133, Advanced Statistics ST2134, ST104A, ST104B, Statistics 1, Statistics 2 no comments

The magic of the distribution function

Let $X$ be a random variable. The function $F_{X}\left( x\right) =P\left(X\leq x\right) ,$ where $x$ runs over real numbers, is called a distribution function of $X.$ In statistics, many formulas are derived with the help of $F_{X}\left( x\right) .$ The motivation and properties are given here

Oftentimes, working with the distribution function is an intermediate step to obtain a density $f_{X}$ using the link

$F_{X}\left( x\right) =\int_{-\infty }^{x}f_{X}\left( t\right) dt.$

A series of exercises below show just how useful the distribution function is.

Exercise 1. Let $Y$ be a linear transformation of $X,$ that is, $Y=\sigma X+\mu ,$ where $\sigma >0$ and $\mu \in R.$ Find the link between $F_{X}$ and $F_{Y}.$ Find the link between $f_{X}$ and $f_{Y}.$
The solution is here.

The more general case of a nonlinear transformation can also be handled:

Exercise 2. Let $Y=g\left( X\right)$ where $g$ is a deterministic function. Suppose that $g$ is strictly monotone, differentiable and $g^{-1}$ exists. Find the link between $F_{X}$ and $F_{Y}.$ Find the link between $f_{X}$ and $f_{Y}.$

Solution. The result differs depending on whether $g$ is increasing or decreasing. Let's assume the latter, so that $x_{1}\leq x_{2}$ is equivalent to $g\left( x_{1}\right) \geq g\left( x_{2}\right) .$ Also for simplicity suppose that $P\left( X=c\right) =0$ for any $c\in R.$ Then

$F_{Y}\left( y\right) =P\left( g\left( X\right) \leq y\right) =P\left( X\geq g^{-1}\left( y\right) \right) =1-P\left( X\leq g^{-1}\left( y\right) \right)=1-F_{X}\left( g^{-1}\left( y\right) \right) .$

Differentiation of this equation produces

$f_{Y}\left( y\right) =-f_{X}\left( g^{-1}\left( y\right) \right) \left( g^{-1}\left( y\right) \right) ^{\prime }=f_{X}\left( g^{-1}\left( y\right) \right) \left\vert \left( g^{-1}\left( y\right) \right) ^{\prime }\right\vert$

(the derivative of $g^{-1}$ is negative).

For an example when $g$ is not invertible see the post about the chi-squared distribution.

Exercise 3. Suppose $T=X/Y$ where $X$ and $Y$ are independent, have densities $f_{X},f_{Y}$ and $Y>0.$ What are the distribution function and density of $T?$

Solution. The joint density $f_{X,Y}$ equals $f_{X}f_{Y},$ so

$F_{T}\left( t\right) =P\left( T\leq t\right) =P\left( X\leq tY\right) =\underset{x\leq ty}{\int \int }f_{X}\left( x\right) f_{Y}\left( y\right) dxdy$

(converting a double integral to an iterated integral and remembering that $f_{Y}$ is zero on the left half-axis)

$=\int_{0}^{\infty }\left( \int_{-\infty }^{ty}f_{X}\left( x\right) dx\right) f_{Y}\left( y\right) dy=\int_{0}^{\infty }F_{X}\left( ty\right) f_{Y}\left( y\right) dy.$

Now by the Leibniz integral rule

(1) $f_{T}\left( t\right) =\int_{0}^{\infty }f_{X}\left( ty\right) yf_{Y}\left( y\right) dy.$

A different method is indicated in Activity 4.11, p.207 of J. Abdey, Guide ST2133.

Exercise 4. Let $X,Y$ be two independent random variables with densities $f_{X},f_{Y}$ . Find $F_{X+Y}$ and $f_{X+Y}.$

See this post.

Exercise 5. Let $X,Y$ be two independent random variables. Find $F_{\max\left\{ X_{1},X_{2}\right\} }$ and $F_{\min \left\{ X_{1},X_{2}\right\} }.$

Solution. The inequality $\max \left\{ X_{1},X_{2}\right\} \leq x$ holds if and only if both $X_{1}\leq x$ and $X_{2}\leq x$ hold. This means that the event $\left\{ \max \left\{ X_{1},X_{2}\right\} \leq x\right\}$ coincides with the event $\left\{ X_{1}\leq x\right\} \cap \left\{ X_{2}\leq x\right\} .$ It follows by independence that

$F_{\max \left\{ X_{1},X_{2}\right\}}\left( x\right) =P\left( \max \left\{ X_{1},X_{2}\right\} \leq x\right) =P\left( \left\{ X_{1}\leq x\right\} \cap \left\{ X_{2}\leq x\right\}\right)$

$=PX_{1}\leq x)P\left( X_{2}\leq x\right) =F_{X_{1}}\left( x\right)F_{X_{2}}\left( x\right) .$

For $\min \left\{ X_{1},X_{2}\right\}$ we need one more trick - pass to the complementary event by writing $F_{\min \left\{ X_{1},X_{2}\right\} }\left( x\right) =P\left( \min \left\{ X_{1},X_{2}\right\} \leq x\right) =1-P\left(\min \left\{ X_{1},X_{2}\right\} >x\right) .$ Now we can use the fact that the event $\left\{ \min \left\{ X_{1},X_{2}\right\} >x\right\}$ coincides with the event $\left\{ X_{1}>x\right\} \cap \left\{ X_{2}>x\right\} .$ Hence, by independence

$F_{\min \left\{ X_{1},X_{2}\right\} }\left( x\right) =1-P\left( \left\{ X_{1}>x\right\} \cap \left\{ X_{2}>x\right\} \right) =1-P\left( X_{1}>x\right) P\left( X_{2}>x\right)$

$=1-\left[ 1-P\left( X_1\leq x\right) \right] \left[ 1-P\left( X_2\leq x\right) \right] =1-\left( 1-F_{X_1}\left( x\right) \right) \left( 1-F_{X_2}\left( x\right) \right) .$

Tags: density of a ratio of two variables, Density of a sum of two variables, distribution function, Distribution function of a linear transformation, distribution function of a maximum of two variables, distribution function of a minimum of two variables, distribution function of a nonlinear transformation, distribution function of a ratio, distribution function of a sum, University of London International Programmes, uol affiliate centre, UoL International Programmes

Dec 22

Final exam in Advanced Statistics ST2133, 2022

category: Advanced Statistics ST2133, Advanced Statistics ST2134, AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2 no comments

Final exam in Advanced Statistics ST2133, 2022

Unlike most UoL exams, here I tried to relate the theory to practical issues.

KBTU International School of Economics

Compiled by Kairat Mynbaev

The total for this exam is 41 points. You have two hours.

Everywhere provide detailed explanations. When answering please clearly indicate question numbers. You don’t need a calculator. As long as the formula you provide is correct, the numerical value does not matter.

Question 1. (12 points)

a) (2 points) At a casino, two players are playing on slot machines. Their payoffs $X,Y$ are standard normal and independent. Find the joint density of the payoffs.

b) (4 points) Two other players watch the first two players and start to argue what will be larger: the sum $U = X + Y$ or the difference $V = X - Y$ . Find the joint density. Are variables $U,V$ independent? Find their marginal densities.

c) (2 points) Are $U,V$ normal? Why? What are their means and variances?

d) (2 points) Which probability is larger: $P(U > V)$ or $P\left( {U < V} \right)$ ?

e) (2 points) In this context interpret the conditional expectation $E\left( {U|V = v} \right)$ . How much is it?

Reminder. The density of a normal variable $X \sim N\left( {\mu ,{\sigma ^2}} \right)$ is ${f_X}\left( x \right) = \frac{1}{{\sqrt {2\pi {\sigma ^2}} }}{e^{ - \frac{{{{\left( {x - \mu } \right)}^2}}}{{2{\sigma ^2}}}}}$ .

Question 2. (9 points) The distribution of a call duration $X$ of one Kcell [largest mobile operator in KZ] customer is exponential: ${f_X}\left( x \right) = \lambda {e^{ - \lambda x}},\,\,x \ge 0,\,\,{f_X}\left( x \right) = 0,\,\,x < 0.$ The number $N$ of customers making calls simultaneously is distributed as Poisson: $P\left( {N = n} \right) = {e^{ - \mu }}\frac{{{\mu ^n}}}{{n!}},\,\,n = 0,1,2,...$ Thus the total call duration for all customers is ${S_N} = {X_1} + ... + {X_N}$ for $N \ge 1$ . We put ${S_0} = 0$ . Assume that customers make their decisions about calling independently.

a) (3 points) Find the general formula (when ${X_1},...,{X_n}$ are identically distributed and $X,N$ are independent but not necessarily exponential and Poisson, as above) for the moment generating function of $S_N$ explaining all steps.

b) (3 points) Find the moment generating functions of $X$ , $N$ and ${S_N}$ for your particular distributions.

c) (3 points) Find the mean and variance of ${S_N}$ . Based on the equations you obtained, can you suggest estimators of parameters $\lambda ,\mu$ ?

Remark. Direct observations on the exponential and Poisson distributions are not available. We have to infer their parameters by observing ${S_N}$ . This explains the importance of the technique used in Question 2.

Question 3. (8 points)

a) (2 points) For a non-negative random variable $X$ prove the Markov inequality $P\left( {X > c} \right) \le \frac{1}{c}EX,\,\,\,c > 0.$

b) (2 points) Prove the Chebyshev inequality $P\left( {|X - EX| > c} \right) \le \frac{1}{c^2}Var\left( X \right)$ for an arbitrary random variable $X$ .

c) (4 points) We say that the sequence of random variables $\left\{ X_n \right\}$ converges in probability to a random variable $X$ if $P\left( {|{X_n} - X| > \varepsilon } \right) \to 0$ as $n \to \infty$ for any $\varepsilon > 0$ . Suppose that $E{X_n} = \mu$ for all $n$ and that $Var\left(X_n \right) \to 0$ as $n \to \infty$ . Prove that then $\left\{X_n\right\}$ converges in probability to $\mu$ .

Remark. Question 3 leads to the simplest example of a law of large numbers: if $\left\{ X_n \right\}$ are i.i.d. with finite variance, then their sample mean converges to their population mean in probability.

Question 4. (8 points)

a) (4 points) Define a distribution function. Give its properties, with intuitive explanations.

b) (4 points) Is a sum of two distribution functions a distribution function? Is a product of two distribution functions a distribution function?

Remark. The answer for part a) is here and the one for part b) is based on it.

Question 5. (4 points) The Rakhat factory prepares prizes for kids for the upcoming New Year event. Each prize contains one type of chocolates and one type of candies. The chocolates and candies are chosen randomly from two production lines, the total number of items is always 10 and all selections are equally likely.

a) (2 points) What proportion of prepared prizes contains three or more chocolates?

b) (2 points) 100 prizes have been sent to an orphanage. What is the probability that 50 of those prizes contain no more than two chocolates?

Tags: Chebyshev inequality, distribution function, exponential distribution, law of large numbers, Markov inequality, moment generating function, normal distribution, Poisson distribution, University of London International Programmes, UoL International Programmes

Oct 22

A problem to do once and never come back

category: Advanced Statistics ST2133, Advanced Statistics ST2134, AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2 no comments

A problem to do once and never come back

There is a problem I gave on the midterm that does not require much imagination. Just know the definitions and do the technical work, so I was hoping we could put this behind us. Turned out we could not and thus you see this post.

Problem. Suppose the joint density of variables $X,Y$ is given by

$f_{X,Y}(x,y)=\left\{\begin{array}{c}k\left( e^{x}+e^{y}\right) \text{ for }0\le y\le x\le 1, \\ 0\text{ otherwise.}\end{array}\right.$

I. Find $k$ .

II. Find marginal densities of $X,Y$ . Are $X,Y$ independent?

III. Find conditional densities $f_{X|Y},\ f_{Y|X}$ .

IV. Find $EX,\ EY$ .

When solving a problem like this, the first thing to do is to give the theory. You may not be able to finish without errors the long calculations but your grade will be determined by the beginning theoretical remarks.

I. Finding the normalizing constant

Any density should satisfy the completeness axiom: the area under the density curve (or in this case the volume under the density surface) must be equal to one: $\int \int f_{X,Y}(x,y)dxdy=1.$ The constant $k$ chosen to satisfy this condition is called a normalizing constant. The integration in general is over the whole plain $R^{2}$ and the first task is to express the above integral as an iterated integral. This is where the domain where the density is not zero should be taken into account. There is little you can do without geometry. One example of how to do this is here.

The shape of the area $A=\left\{ (x,y):0<y<x<1\right\}$ is determined by a) the extreme values of $x,y$ and b) the relationship between them. The extreme values are 0 and 1 for both $x$ and $y$ , meaning that $A$ is contained in the square $\left\{ (x,y):0<x,y\text{ and}\ x,y<1\right\} .$ The inequality $y<x$ means that we cut out of this square the triangle below the line $y=x$ (it is really the lower triangle because if from a point on the line $y=x$ we move down vertically, $x$ will stay the same and $y$ will become smaller than $x$ ).

In the iterated integral:

a) the lower and upper limits of integration for the inner integral are the boundaries for the inner variable; they may depend on the outer variable but not on the inner variable.

b) the lower and upper limits of integration for the outer integral are the extreme values for the outer variable; they must be constant.

This is illustrated in Pane A of Figure 1.

Figure 1. Integration order

Always take the inner integral in parentheses to show that you are dealing with an iterated integral.

a) In the inner integral integrating over $x$ means moving along blue arrows from the boundary $x=y$ to the boundary $x=1.$ The boundaries may depend on $y$ but not on $x$ because the outer integral is over $y.$

b) In the outer integral put the extreme values for the outer variable. Thus,

$\underset{A}{\int \int }f_{X,Y}(x,y)dxdy=\int_{0}^{1}\left(\int_{y}^{1}f_{X,Y}(x,y)dx\right) dy.$

Check that if we first integrate over $y$ (vertically along red arrows, see Pane B in Figure 1) then the equation

$\underset{A}{\int \int }f_{X,Y}(x,y)dxdy=\int_{0}^{1}\left(\int_{0}^{x}f_{X,Y}(x,y)dy\right) dx$

results.

In fact, from the definition $A=\left\{ (x,y):0<y<x<1\right\}$ one can see that the inner interval for $x$ is $\left[ y,1\right]$ and for $y$ it is $\left[ 0,x\right] .$

II. Marginal densities

I can't say about this more than I said here.

The condition for independence of $X,Y$ is

$f_{X,Y}\left(x,y\right)=f_{X}\left( x\right) f_{Y}\left(y\right)$

(this is a direct analog of the independence condition for events $P\left(A\cap B\right) =P\left(A\right) P\left(B\right)$ ). In words: the joint density decomposes into a product of individual densities.

III. Conditional densities

In this case the easiest is to recall the definition of conditional probability $P\left( A|B\right) =\frac{P\left( A\cap B\right) }{P\left(B\right) }.$ The definition of conditional densities $f_{X|Y},\ f_{Y|X}$ is quite similar:

(2) $f_{X|Y}\left( x|y\right) =\frac{f_{X,Y}\left( x,y\right) }{f_{Y}\left(y\right) },\ f_{Y|X}\left( y|x\right) =\frac{f_{X,Y}\left( x,y\right) }{f_{X}\left( x\right) }$ .

Of course, $f_{Y}\left( y\right) ,f_{X}\left( x\right)$ here can be replaced by their marginal equivalents.

IV. Finding expected values of $X,Y$

The usual definition $EX=\int xf_{X}\left( x\right) dx$ takes an equivalent form using the marginal density:

$EX=\int x\left( \int f_{X,Y}\left( x,y\right) dy\right) dx=\int \int xf_{X,Y}\left( x,y\right) dydx.$

Which equation to use is a matter of convenience.

Another replacement in the usual definition gives the definition of conditional expectations:

$E\left( X|Y\right) =\int xf_{X|Y}\left( x|y\right) dx,$

$E\left( Y|X\right) =\int yf_{Y|X}\left( y|x\right) dx.$

Note that these are random variables: $E\left( X|Y=y\right)$ depends in $y$ and $E\left( Y|X=x\right)$ depends on $x.$

Solution to the problem

Being a lazy guy, for the problem this post is about I provide answers found in Mathematica:

I. $k=0.581977$

II. $f_{X}\left( x\right) =-1+e^{x}\left( 1+x\right) ,$ for $x\in[ 0,1],$ $f_{Y}\left( y\right) =e-e^{y}y,$ for $y\in \left[ 0,1\right] .$

It is readily seen that the independence condition is not satisfied.

III. $f_{X|Y}\left( x|y\right) =\frac{k\left( e^{x}+e^{y}\right) }{e-e^{y}y}$ for $0<y<x<1,$

$f_{Y|X}\left(y|x\right) =\frac{k\left(e^x+e^y\right) }{-1+e^x\left( 1+x\right) }$ for $0<y<x<1.$

IV. $EX=0.709012,$ $EY=0.372965.$

Tags: conditional density, iterated integral, joint density, Marginal density, University of London International Programmes, uol affiliate centre

Oct 22

Marginal probabilities and densities

category: Advanced Statistics ST2133, Advanced Statistics ST2134, AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2 no comments

Marginal probabilities and densities

This is to help everybody, from those who study Basic Statistics up to Advanced Statistics ST2133.

Discrete case

Suppose in a box we have coins and banknotes of only two denominations: $1 and $5 (see Figure 1).

Figure 1. Illustration of two variables

We pull one out randomly. The division of cash by type (coin or banknote) divides the sample space (shown as a square, lower left picture) with probabilities $p_{c}$ and $p_{b}$ (they sum to one). The division by denomination ($1 or $5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and $5 equal to $p_{1}$ and $p_{5}$ , resp. (they also sum to one). This is summarized in the tables

Variable 1: Cash type	Prob
coin	$p_{c}$
banknote	$p_{b}$

Variable 2: Denomination	Prob
$1	$p_{1}$
$5	$p_{5}$

Now we can consider joint events and probabilities (see Figure 2, where the two divisions are combined).

Figure 2. Joint probabilities

For example, if we pull out a random $item$ it can be a $coin$ and $1 and the corresponding probability is $P\left(item=coin,\ item\ value=\$1\right) =p_{c1}.$ The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:

Adding over denominations: $p_{c1}+p_{c5}=p_{c},$ $p_{b1}+p_{b5}=p_{b},$

Adding over cash types: $p_{c1}+p_{b1}=p_{1},$ $p_{c5}+p_{b5}=p_{5}.$

Formally, here we use additivity of probability for disjoint events

$P\left( A\cup B\right) =P\left( A\right) +P\left( B\right) .$

In words: we can recover own probabilities of variables 1,2 from joint probabilities.

Generalization

Suppose we have two discrete random variables $X,Y$ taking values $x_{1},...,x_{n}$ and $y_{1},...,y_{m},$ resp., and their own probabilities are $P\left( X=x_{i}\right) =p_{i}^{X},$ $P\left(Y=y_{j}\right) =p_{j}^{Y}.$ Denote the joint probabilities $P\left(X=x_{i},Y=y_{j}\right) =p_{ij}.$ Then we have the identities

(1) $\sum_{j=1}^mp_{ij}=p_{i}^{X},$ $\sum_{i=1}^np_{ij}=p_{j}^{Y}$ ( $n+m$ equations).

In words: to obtain the marginal probability of one variable (say, $Y$ ) sum over the values of the other variable (in this case, $X$ ).

The name marginal probabilities is used for $p_{i}^{X},p_{j}^{Y}$ because in the two-dimensional table they arise as a result of summing table entries along columns or rows and are displayed in the margins.

Analogs for continuous variables with densities

Suppose we have two continuous random variables $X,Y$ and their own densities are $f_{X}$ and $f_{Y}.$ Denote the joint density $f_{X,Y}$ . Then replacing in (1) sums by integrals and probabilities by densities we get

(2) $\int_R f_{X,Y}\left( x,y\right) dy=f_{X}\left( x\right) ,\ \int_R f_{X,Y}\left( x,y\right) dx=f_{Y}\left( y\right) .$

In words: to obtain one marginal density (say, $f_{Y}$ ) integrate out the other variable (in this case, $x$ ).

Tags: AP Statistics, Business Statistics, Marginal density, Marginal probability, University of London International Programmes, uol affiliate centre

Jan 21

My book is gaining international recognition

category: AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2, The College Board no comments

AP Stats and Business Stats

Its content, organization and level justify its adoption as a textbook for introductory statistics for Econometrics in most American or European universities. The book's table of contents is somewhat standard, the innovation comes in a presentation that is crisp, concise, precise and directly relevant to the Econometrics course that will follow. I think instructors and students will appreciate the absence of unnecessary verbiage that permeates many existing textbooks.

Having read Professor Mynbaev's previous books and research articles I was not surprised with his clear writing and precision. However, I was surprised with an informal and almost conversational one-on-one style of writing which should please most students. The informality belies a careful presentation where great care has been taken to present the material in a pedagogical manner.

Carlos Martins-Filho
Professor of Economics
University of Colorado at Boulder
Boulder, USA

Tags: AP Statistics, Business Statistics, simple regression, University of London International Programmes, uol affiliate centre, UoL International Programmes

May 20

My book in Basic Statistics

category: AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2, The College Board no comments

Number of reads of my book on October 29, 2020

For more information see my site

Tags: ANOVA, AP Statistics, Business Statistics, discrete and continuous variables, estimation, graphical and numerical description of data, hypothesis testing, Maximum likelihood method, sampling and sampling distribution, simple regression, University of London International Programmes, uol affiliate centre, UoL International Programmes

Mar 20

Statistical calculator

category: AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2, The College Board no comments

Statistical calculator

In my book I explained how one can use Excel to do statistical simulations and replace statistical tables commonly used in statistics courses. Here I go one step further by providing a free statistical calculator that replaces the following tables from the book by Newbold et al.:

Table 1 Cumulative Distribution Function, F(z), of the Standard Normal Distribution Table

Table 2 Probability Function of the Binomial Distribution

Table 5 Individual Poisson Probabilities

Table 7a Upper Critical Values of Chi-Square Distribution with $\nu$ Degrees of Freedom

Table 8 Upper Critical Values of Student’s t Distribution with $\nu$ Degrees of Freedom

Tables 9a, 9b Upper Critical Values of the F Distribution

The calculator is just a Google sheet with statistical functions, see Picture 1:

Picture 1. Calculator using Google sheet

How to use Calculator

Open an account at gmail.com, if you haven't already. Open Google Drive.
Install Google sheets on your phone.
Find the sheet on my Google drive and copy it to your Google drive (File/Make a copy). An icon of my calculator will appear in your drive. That's not the file, it's just a link to my file. To the right of it there are three dots indicating options. One of them is "Make a copy", so use that one. The copy will be in your drive. After that you can delete the link to my file. You might want to rename "Copy of Calculator" as "Calculator".
Open the file on your drive using Google sheets. Your Calculator is ready!
When you click a cell, you can enter what you need either in the formula bar at the bottom or directly in the cell. You can also see the functions I embedded in the sheet.
In cell A1, for example, you can enter any legitimate formula with numbers, arithmetic signs, and Google sheet functions. Be sure to start it with =,+ or - and to press the checkmark on the right of the formula bar after you finish.
The cells below A1 replace the tables listed above. Beside each function there is a verbal description and further to the right - a graphical illustration (which is not in Picture 1).
On the tab named Regression you can calculate the slope and intercept. The sample size must be 10.
Keep in mind that tables for continuous distributions need two functions. For example, in case of the standard normal distribution one function allows you to go from probability (area of the left tail) to the cutting value on the horizontal axis. The other function goes from the cutting value on the horizontal axis to probability.
Feel free to add new sheets or functions as you may need. You will have to do this on a tablet or computer.

Tags: Statistical calculator

Mar 19

AP Statistics the Genghis Khan way 2

category: AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2, The College Board no comments

AP Statistics the Genghis Khan way 2

Last semester I tried to explain theory through numerical examples. The results were terrible. Even the best students didn't stand up to my expectations. The midterm grades were so low that I did something I had never done before: I allowed my students to write an analysis of the midterm at home. Those who were able to verbally articulate the answers to me received a bonus that allowed them to pass the semester.

This semester I made a U-turn. I announced that in the first half of the semester we will concentrate on theory and we followed this methodology. Out of 35 students, 20 significantly improved their performance and 15 remained where they were.

Midterm exam, version 1

1. General density definition (6 points)

a. Define the density $p_X$ of a random variable $X.$ Draw the density of heights of adults, making simplifying assumptions if necessary. Don't forget to label the axes.

b. According to your plot, how much is the integral $\int_{-\infty}^0p_X(t)dt?$ Explain.

c. Why the density cannot be negative?

d. Why the total area under the density curve should be 1?

e. Where are basketball players on your graph? Write down the corresponding expression for probability.

f. Where are dwarfs on your graph? Write down the corresponding expression for probability.

This question is about the interval formula. In each case students have to write the equation for the probability and the corresponding integral of the density. At this level, I don't talk about the distribution function and introduce the density by the interval formula.

2. Properties of means (8 points)

a. Define a discrete random variable and its mean.

b. Define linear operations with random variables.

c. Prove linearity of means.

d. Prove additivity and homogeneity of means.

e. How much is the mean of a constant?

f. Using induction, derive the linearity of means for the case of $n$ variables from the case of two variables (3 points).

3. Covariance properties (6 points)

a. Derive linearity of covariance in the first argument when the second is fixed.

b. How much is covariance if one of its arguments is a constant?

c. What is the link between variance and covariance? If you know one of these functions, can you find the other (there should be two answers)? (4 points)

4. Standard normal variable (6 points)

a. Define the density $p_z(t)$ of a standard normal.

b. Why is the function $p_z(t)$ even? Illustrate this fact on the plot.

c. Why is the function $f(t)=tp_z(t)$ odd? Illustrate this fact on the plot.

d. Justify the equation $Ez=0.$

e. Why is $V(z)=1?$

f. Let $t>0.$ Show on the same plot areas corresponding to the probabilities $A_1=P(0<z<t),$ $A_2=P(z>t),$ $A_3=P(z<-t),$ $A_4=P(-t<z<0).$ Write down the relationships between $A_1,...,A_4.$

5. General normal variable (3 points)

a. Define a general normal variable $X.$

b. Use this definition to find the mean and variance of $X.$

c. Using part b, on the same plot graph the density of the standard normal and of a general normal with parameters $\sigma =2,$ $\mu =3.$

Midterm exam, version 2

1. General density definition (6 points)

a. Define the density $p_X$ of a random variable $X.$ Draw the density of work experience of adults, making simplifying assumptions if necessary. Don't forget to label the axes.

b. According to your plot, how much is the integral $\int_{-\infty}^0p_X(t)dt?$ Explain.

c. Why the density cannot be negative?

d. Why the total area under the density curve should be 1?

e. Where are retired people on your graph? Write down the corresponding expression for probability.

f. Where are young people (up to 25 years old) on your graph? Write down the corresponding expression for probability.

2. Variance properties (8 points)

a. Define variance of a random variable. Why is it non-negative?

b. Define the formula for variance of a linear combination of two variables.

c. How much is variance of a constant?

d. What is the formula for variance of a sum? What do we call homogeneity of variance?

e. What is larger: $V(X+Y)$ or $V(X-Y)$ ? (2 points)

f. One investor has 100 shares of Apple, another - 200 shares. Which investor's portfolio has larger variability? (2 points)

3. Poisson distribution (6 points)

a. Write down the Taylor expansion and explain the idea. How are the Taylor coefficients found?

b. Use the Taylor series for the exponential function to define the Poisson distribution.

c. Find the mean of the Poisson distribution. What is the interpretation of the parameter $\lambda$ in practice?

4. Standard normal variable (6 points)

a. Define the density $p_z(t)$ of a standard normal.

b. Why is the function $p_z(t)$ even? Illustrate this fact on the plot.

c. Why is the function $f(t)=tp_z(t)$ odd? Illustrate this fact on the plot.

d. Justify the equation $Ez=0.$

e. Why is $V(z)=1?$

f. Let $t>0.$ Show on the same plot areas corresponding to the probabilities $A_1=P(0<z<t),$ $A_2=P(z>t),$ $A_{3}=P(z<-t),$ $A_4=P(-t<z<0).$ Write down the relationships between $A_{1},...,A_{4}.$

5. General normal variable (3 points)

a. Define a general normal variable $X.$

b. Use this definition to find the mean and variance of $X.$

c. Using part b, on the same plot graph the density of the standard normal and of a general normal with parameters $\sigma =2,$ $\mu =3.$

Tags: AP Statistics, Business Statistics, midterm exam, University of London International Programmes, uol affiliate centre

Nov 18

Little tricks for AP Statistics

category: AP Stats and Business Stats, ST104A, ST104B, Statistics 1, Statistics 2, The College Board no comments

Little tricks for AP Statistics

This year I am teaching AP Statistics. If the things continue the way they are, about half of the class will fail. Here is my diagnosis and how I am handling the problem.

On the surface, the students lack algebra training but I think the problem is deeper: many of them have underdeveloped cognitive abilities. Their perception is slow, memory is limited, analytical abilities are rudimentary and they are not used to work at home. Limited resources require careful allocation.

Terminology

Short and intuitive names are better than two-word professional names.

Instead of "sample space" or "probability space" say "universe". The universe is the widest possible event, and nothing exists outside it.

Instead of "elementary event" say "atom". Simplest possible events are called atoms. This corresponds to the theoretical notion of an atom in measure theory (an atom is a measurable set which has positive measure and contains no set of smaller positive measure).

Then the formulation of classical probability becomes short. Let $n$ denote the number of atoms in the universe and let $n_A$ be the number of atoms in event $A.$ If all atoms are equally likely (have equal probabilities), then $P(A)=n_A/n.$

The clumsy "mutually exclusive events" are better replaced by more visual "disjoint sets". Likewise, instead of "collectively exhaustive events" say "events that cover the universe".

The combination "mutually exclusive" and "collectively exhaustive" events is beyond comprehension for many. I say: if events are disjoint and cover the universe, we call them tiles. To support this definition, play onscreen one of jigsaw puzzles (Video 1) and produce the picture from Figure 1.

Video 1. Tiles (disjoint events that cover the universe)

Figure 1. Tiles (disjoint events that cover the universe)

The philosophy of team work

We are in the same boat. I mean the big boat. Not the class. Not the university. It's the whole country. We depend on each other. Failure of one may jeopardize the well-being of everybody else.

You work in teams. You help each other to learn. My lectures and your presentations are just the beginning of the journey of knowledge into your heads. I cannot control how it settles there. Be my teaching assistants, share your big and little discoveries with your classmates.

I don't just preach about you helping each other. I force you to work in teams. 30% of the final grade is allocated to team work. Team work means joint responsibility. You work on assignments together. I randomly select a team member for reporting. His or her grade is what each team member gets.

This kind of team work is incompatible with the Western obsession with grades privacy. If I say my grade is nobody's business, by extension I consider the level of my knowledge a private issue. This will prevent me from asking for help and admitting my errors. The situation when students hide their errors and weaknesses from others also goes against the ethics of many workplaces. In my class all grades are public knowledge.

In some situations, keeping the grade private is technically impossible. Conducting a competition without announcing the points won is impossible. If I catch a student cheating, I announce the failing grade immediately, as a warning to others.

To those of you who think team-based learning is unfair to better students I repeat: 30% of the final grade is given for team work, not for personal achievements. The other 70% is where you can shine personally.

Breaking the wall of silence

Team work serves several purposes.

Firstly, joint responsibility helps breaking communication barriers. See in Video 2 my students working in teams on classroom assignments. The situation when a weaker student is too proud to ask for help and a stronger student doesn't want to offend by offering help is not acceptable. One can ask for help or offer help without losing respect for each other.

Video 2. Teams working on assignments

Secondly, it turns on resources that are otherwise idle. Explaining something to somebody is the best way to improve your own understanding. The better students master a kind of leadership that is especially valuable in a modern society. For the weaker students, feeling responsible for a team improves motivation.

Thirdly, I save time by having to grade less student papers.

On exams and quizzes I mercilessly punish the students for Yes/No answers without explanations. There are no half-points for half-understanding. This, in combination with the team work and open grades policy allows me to achieve my main objective: students are eager to talk to me about their problems.

Set operations and probability

After studying the basics of set operations and probabilities we had a midterm exam. It revealed that about one-third of students didn't understand this material and some of that misunderstanding came from high school. During the review session I wanted to see if they were ready for a frank discussion and told them: "Those who don't understand probabilities, please raise your hands", and about one-third raised their hands. I invited two of them to work at the board.

Video 3. Translating verbal statements to sets, with accompanying probabilities

Many teachers think that the Venn diagrams explain everything about sets because they are visual. No, for some students they are not visual enough. That's why I prepared a simple teaching aid (see Video 3) and explained the task to the two students as follows:

I am shooting at the target. The target is a square with two circles on it, one red and the other blue. The target is the universe (the bullet cannot hit points outside it). The probability of a set is its area. I am going to tell you one statement after another. You write that statement in the first column of the table. In the second column write the mathematical expression for the set. In the third column write the probability of that set, together with any accompanying formulas that you can come up with. The formulas should reflect the relationships between relevant areas.

Table 1. Set operations and probabilities

Statement	Set	Probability
1. The bullet hit the universe	$S$	$P(S)=1$
2. The bullet didn't hit the universe	$\emptyset$	$P(\emptyset )=0$
3. The bullet hit the red circle	$A$	$P(A)$
4. The bullet didn't hit the red circle	$\bar{A}=S\backslash A$	$P(\bar{A})=P(S)-P(A)=1-P(A)$
5. The bullet hit both the red and blue circles	$A\cap B$	$P(A\cap B)$ (in general, this is not equal to $P(A)P(B)$ )
6. The bullet hit $A$ or $B$ (or both)	$A\cup B$	$P(A\cup B)=P(A)+P(B)-P(A\cap B)$ (additivity rule)
7. The bullet hit $A$ but not $B$	$A\backslash B$	$P(A\backslash B)=P(A)-P(A\cap B)$
8. The bullet hit $B$ but not $A$	$B\backslash A$	$P(B\backslash A)=P(B)-P(A\cap B)$
9. The bullet hit either $A$ or $B$ (but not both)	$(A\backslash B)\cup(B\backslash A)$	$P\left( (A\backslash B)\cup (B\backslash A)\right)$ $=P(A)+P(B)-2P(A\cap B)$

During the process, I was illustrating everything on my teaching aid. This exercise allows the students to relate verbal statements to sets and further to their areas. The main point is that people need to see the logic, and that logic should be repeated several times through similar exercises.

Tags: atom, classical probability, difference of events, disjoint sets, elementary event, equally likely events, intersection of events, probability space, sample space, team-based learning, tiles, union of events, universe, University of London International Programmes, uol affiliate centre

Oct 17

Reevaluating probabilities based on piece of evidence

category: AP Stats and Business Stats, FN3142 Quantitative Finance, ST104A, ST104B, Statistics 1, Statistics 2, The College Board no comments

Reevaluating probabilities based on piece of evidence

This actually has to do with the Bayes' theorem. However, in simple problems one can use a dead simple approach: just find probabilities of all elementary events. This post builds upon the post on Significance level and power of test, including the notation. Be sure to review that post.

Here is an example from the guide for Quantitative Finance by A. Patton (University of London course code FN3142).

Activity 7.2 Consider a test that has a Type I error rate of 5%, and power of 50%.

Suppose that, before running the test, the researcher thinks that both the null and the alternative are equally likely.

If the test indicates a rejection of the null hypothesis, what is the probability that the null is false?
If the test indicates a failure to reject the null hypothesis, what is the probability that the null is true?

Denote events R = {Reject null}, A = {fAil to reject null}; T = {null is True}; F = {null is False}. Then we are given:

(1) $P(F)=0.5;\ P(T)=0.5;$

(2) $P(R|T)=\frac{P(R\cap T)}{P(T)}=0.05;\ P(R|F)=\frac{P(R\cap F)}{P(F)}=0.5;$

(1) and (2) show that we can find $P(R\cap T)$ and $P(R\cap F)$ and therefore also $P(A\cap T)$ and $P(A\cap F).$ Once we know probabilities of elementary events, we can find everything about everything.