27
Dec 22

## Final exam in Advanced Statistics ST2133, 2022

Unlike most UoL exams, here I tried to relate the theory to practical issues.

KBTU International School of Economics

Compiled by Kairat Mynbaev

The total for this exam is 41 points. You have two hours.

Everywhere provide detailed explanations. When answering please clearly indicate question numbers. You don’t need a calculator. As long as the formula you provide is correct, the numerical value does not matter.

Question 1. (12 points)

a) (2 points) At a casino, two players are playing on slot machines. Their payoffs $X,Y$ are standard normal and independent. Find the joint density of the payoffs.

b) (4 points) Two other players watch the first two players and start to argue what will be larger: the sum $U = X + Y$ or the difference $V = X - Y$. Find the joint density. Are variables $U,V$ independent? Find their marginal densities.

c) (2 points) Are $U,V$ normal? Why? What are their means and variances?

d) (2 points) Which probability is larger: $P(U > V)$ or $P\left( {U < V} \right)$?

e) (2 points) In this context interpret the conditional expectation $E\left( {U|V = v} \right)$. How much is it?

Reminder. The density of a normal variable $X \sim N\left( {\mu ,{\sigma ^2}} \right)$ is ${f_X}\left( x \right) = \frac{1}{{\sqrt {2\pi {\sigma ^2}} }}{e^{ - \frac{{{{\left( {x - \mu } \right)}^2}}}{{2{\sigma ^2}}}}}$.

Question 2. (9 points) The distribution of a call duration $X$ of one Kcell [largest mobile operator in KZ] customer is exponential: ${f_X}\left( x \right) = \lambda {e^{ - \lambda x}},\,\,x \ge 0,\,\,{f_X}\left( x \right) = 0,\,\,x < 0.$ The number $N$ of customers making calls simultaneously is distributed as Poisson: $P\left( {N = n} \right) = {e^{ - \mu }}\frac{{{\mu ^n}}}{{n!}},\,\,n = 0,1,2,...$ Thus the total call duration for all customers is ${S_N} = {X_1} + ... + {X_N}$ for $N \ge 1$. We put ${S_0} = 0$. Assume that customers make their decisions about calling independently.

a) (3 points) Find the general formula (when ${X_1},...,{X_n}$ are identically distributed and $X,N$ are independent but not necessarily exponential and Poisson, as above) for the moment generating function of $S_N$ explaining all steps.

b) (3 points) Find the moment generating functions of $X$, $N$ and ${S_N}$ for your particular distributions.

c) (3 points) Find the mean and variance of ${S_N}$. Based on the equations you obtained, can you suggest estimators of parameters $\lambda ,\mu$?

Remark. Direct observations on the exponential and Poisson distributions are not available. We have to infer their parameters by observing ${S_N}$. This explains the importance of the technique used in Question 2.

Question 3. (8 points)

a) (2 points) For a non-negative random variable $X$ prove the Markov inequality $P\left( {X > c} \right) \le \frac{1}{c}EX,\,\,\,c > 0.$

b) (2 points) Prove the Chebyshev inequality $P\left( {|X - EX| > c} \right) \le \frac{1}{c^2}Var\left( X \right)$ for an arbitrary random variable $X$.

c) (4 points) We say that the sequence of random variables $\left\{ X_n \right\}$ converges in probability to a random variable $X$ if $P\left( {|{X_n} - X| > \varepsilon } \right) \to 0$ as $n \to \infty$ for any $\varepsilon > 0$.  Suppose that $E{X_n} = \mu$ for all $n$ and that $Var\left(X_n \right) \to 0$ as $n \to \infty$. Prove that then $\left\{X_n\right\}$ converges in probability to $\mu$.

Remark. Question 3 leads to the simplest example of a law of large numbers: if $\left\{ X_n \right\}$ are i.i.d. with finite variance, then their sample mean converges to their population mean in probability.

Question 4. (8 points)

a) (4 points) Define a distribution function. Give its properties, with intuitive explanations.

b) (4 points) Is a sum of two distribution functions a distribution function? Is a product of two distribution functions a distribution function?

Remark. The answer for part a) is here and the one for part b) is based on it.

Question 5. (4 points) The Rakhat factory prepares prizes for kids for the upcoming New Year event. Each prize contains one type of chocolates and one type of candies. The chocolates and candies are chosen randomly from two production lines, the total number of items is always 10 and all selections are equally likely.

a) (2 points) What proportion of prepared prizes contains three or more chocolates?

b) (2 points) 100 prizes have been sent to an orphanage. What is the probability that 50 of those prizes contain no more than two chocolates?

24
Oct 22

## A problem to do once and never come back

There is a problem I gave on the midterm that does not require much imagination. Just know the definitions and do the technical work, so I was hoping we could put this behind us. Turned out we could not and thus you see this post.

Problem. Suppose the joint density of variables $X,Y$ is given by

$f_{X,Y}(x,y)=\left\{ \begin{array}{c}k\left( e^{x}+e^{y}\right) \text{ for }0

I. Find $k$.

II. Find marginal densities of $X,Y$. Are $X,Y$ independent?

III. Find conditional densities $f_{X|Y},\ f_{Y|X}$.

IV. Find $EX,\ EY$.

When solving a problem like this, the first thing to do is to give the theory. You may not be able to finish without errors the long calculations but your grade will be determined by the beginning theoretical remarks.

### I. Finding the normalizing constant

Any density should satisfy the completeness axiom: the area under the density curve (or in this case the volume under the density surface) must be equal to one: $\int \int f_{X,Y}(x,y)dxdy=1.$ The constant $k$ chosen to satisfy this condition is called a normalizing constant. The integration in general is over the whole plain $R^{2}$ and the first task is to express the above integral as an iterated integral. This is where the domain where the density is not zero should be taken into account. There is little you can do without geometry. One example of how to do this is here.

The shape of the area $A=\left\{ (x,y):0 is determined by a) the extreme values of $x,y$ and b) the relationship between them. The extreme values are 0 and 1 for both $x$ and $y$, meaning that $A$ is contained in the square $\left\{ (x,y):0 The inequality $y means that we cut out of this square the triangle below the line $y=x$ (it is really the lower triangle because if from a point on the line $y=x$ we move down vertically, $x$ will stay the same and $y$ will become smaller than $x$).

In the iterated integral:

a) the lower and upper limits of integration for the inner integral are the boundaries for the inner variable; they may depend on the outer variable but not on the inner variable.

b) the lower and upper limits of integration for the outer integral are the extreme values for the outer variable; they must be constant.

This is illustrated in Pane A of Figure 1.

Figure 1. Integration order

Always take the inner integral in parentheses to show that you are dealing with an iterated integral.

a) In the inner integral integrating over $x$ means moving along blue arrows from the boundary $x=y$ to the boundary $x=1.$ The boundaries may depend on $y$ but not on $x$ because the outer integral is over $y.$

b) In the outer integral put the extreme values for the outer variable. Thus,

$\underset{A}{\int \int }f_{X,Y}(x,y)dxdy=\int_{0}^{1}\left(\int_{y}^{1}f_{X,Y}(x,y)dx\right) dy.$

Check that if we first integrate over $y$ (vertically along red arrows, see Pane B in Figure 1) then the equation

$\underset{A}{\int \int }f_{X,Y}(x,y)dxdy=\int_{0}^{1}\left(\int_{0}^{x}f_{X,Y}(x,y)dy\right) dx$

results.

In fact, from the definition $A=\left\{ (x,y):0 one can see that the inner interval for $x$ is $\left[ y,1\right]$ and for $y$ it is $\left[ 0,x\right] .$

### II. Marginal densities

The condition for independence of $X,Y$ is $f_{X,Y}\left( x,y\right) =f_{X}\left( x\right) f_{Y}\left( y\right)$ (this is a direct analog of the independence condition for events $P\left( A\cap B\right) =P\left( A\right) P\left( B\right)$). In words: the joint density decomposes into a product of individual densities.

### III. Conditional densities

In this case the easiest is to recall the definition of conditional probability $P\left( A|B\right) =\frac{P\left( A\cap B\right) }{P\left(B\right) }.$ The definition of conditional densities $f_{X|Y},\ f_{Y|X}$ is quite similar:

(2) $f_{X|Y}\left( x|y\right) =\frac{f_{X,Y}\left( x,y\right) }{f_{Y}\left( y\right) },\ f_{Y|X}\left( y|x\right) =\frac{f_{X,Y}\left( x,y\right) }{f_{X}\left( x\right) }$.

Of course, $f_{Y}\left( y\right) ,f_{X}\left( x\right)$ here can be replaced by their marginal equivalents.

### IV. Finding expected values of $X,Y$$X,Y$

The usual definition $EX=\int xf_{X}\left( x\right) dx$ takes an equivalent form using the marginal density:

$EX=\int x\left( \int f_{X,Y}\left( x,y\right) dy\right) dx=\int \int xf_{X,Y}\left( x,y\right) dydx.$

Which equation to use is a matter of convenience.

Another replacement in the usual definition gives the definition of conditional expectations:

$E\left( X|Y\right) =\int xf_{X|Y}\left( x|y\right) dx,$ $E\left( Y|X\right) =\int yf_{Y|X}\left( y|x\right) dx.$

Note that these are random variables: $E\left( X|Y=y\right)$ depends in $y$ and $E\left( Y|X=x\right)$ depends on $x.$

### Solution to the problem

Being a lazy guy, for the problem this post is about I provide answers found in Mathematica:

I. $k=0.581977$

II. $f_{X}\left( x\right) =-1+e^{x}\left( 1+x\right) ,$ for $x\in[ 0,1],$ $f_{Y}\left( y\right) =e-e^{y}y,$ for $y\in \left[ 0,1\right] .$

It is readily seen that the independence condition is not satisfied.

III. $f_{X|Y}\left( x|y\right) =\frac{k\left( e^{x}+e^{y}\right) }{e-e^{y}y}$ for $0

$f_{Y|X}\left(y|x\right) =\frac{k\left(e^x+e^y\right) }{-1+e^x\left( 1+x\right) }$ for $0

IV. $EX=0.709012,$ $EY=0.372965.$

24
Oct 22

## Marginal probabilities and densities

This is to help everybody, from those who study Basic Statistics up to Advanced Statistics ST2133.

### Discrete case

Suppose in a box we have coins and banknotes of only two denominations: $1 and$5 (see Figure 1).

Figure 1. Illustration of two variables

We pull one out randomly. The division of cash by type (coin or banknote) divides the sample space (shown as a square, lower left picture) with probabilities $p_{c}$ and $p_{b}$ (they sum to one). The division by denomination ($1 or$5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and$5 equal to $p_{1}$ and $p_{5}$, resp. (they also sum to one). This is summarized in the tables

 Variable 1: Cash type Prob coin $p_{c}$$p_{c}$ banknote $p_{b}$$p_{b}$
 Variable 2: Denomination Prob $1 $p_{1}$$p_{1}$$5 $p_{5}$$p_{5}$

Now we can consider joint events and probabilities (see Figure 2, where the two divisions are combined).

Figure 2. Joint probabilities

For example, if we pull out a random $item$ it can be a $coin$ and \$1 and the corresponding probability is $P\left(item=coin,\ item\ value=\1\right) =p_{c1}.$ The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:

Adding over denominations: $p_{c1}+p_{c5}=p_{c},$ $p_{b1}+p_{b5}=p_{b},$

Adding over cash types: $p_{c1}+p_{b1}=p_{1},$ $p_{c5}+p_{b5}=p_{5}.$

Formally, here we use additivity of probability for disjoint events

$P\left( A\cup B\right) =P\left( A\right) +P\left( B\right) .$

In words: we can recover own probabilities of variables 1,2 from joint probabilities.

### Generalization

Suppose we have two discrete random variables $X,Y$ taking values $x_{1},...,x_{n}$ and $y_{1},...,y_{m},$ resp., and their own probabilities are $P\left( X=x_{i}\right) =p_{i}^{X},$ $P\left(Y=y_{j}\right) =p_{j}^{Y}.$ Denote the joint probabilities $P\left(X=x_{i},Y=y_{j}\right) =p_{ij}.$ Then we have the identities

(1) $\sum_{j=1}^mp_{ij}=p_{i}^{X},$ $\sum_{i=1}^np_{ij}=p_{j}^{Y}$ ($n+m$ equations).

In words: to obtain the marginal probability of one variable (say, $Y$) sum over the values of the other variable (in this case, $X$).

The name marginal probabilities is used for $p_{i}^{X},p_{j}^{Y}$ because in the two-dimensional table they arise as a result of summing table entries along columns or rows and are displayed in the margins.

### Analogs for continuous variables with densities

Suppose we have two continuous random variables $X,Y$ and their own densities are $f_{X}$ and $f_{Y}.$ Denote the joint density $f_{X,Y}$. Then replacing in (1) sums by integrals and probabilities by densities we get

(2) $\int_R f_{X,Y}\left( x,y\right) dy=f_{X}\left( x\right) ,\ \int_R f_{X,Y}\left( x,y\right) dx=f_{Y}\left( y\right) .$

In words: to obtain one marginal density (say, $f_{Y}$) integrate out the other variable (in this case, $x$).

25
Dec 21

## Analysis of problems with conditioning

These problems are among the most difficult. It's important to work out a general approach to such problems. All references are to J. Abdey,  Advanced statistics: distribution theory, ST2133, University of London, 2021.

### General scheme

Step 1. Conditioning is usually suggested by the problem statement: $Y$ is conditioned on $X$.

Your life will be easier if you follow the notation used in the guide: use $p$ for probability mass functions (discrete variables) and $f$ for (probability) density functions (continuous variables).

a) If $Y|X$ and $X$ both are discrete (Example 5.1, Example 5.13, Example 5.18):

$p_{Y}\left( y\right) =\sum_{Set}p_{Y\vert X}\left( y\vert x\right) p_{X}\left( x\right) .$

b) If $Y|X$ and $X$ both are continuous (Activity 5.6):

$f_{Y}\left( y\right) =\int_{Set}f_{Y\vert X}\left( y\vert x\right) f_{X}\left( x\right) dx.$

c) If $Y|X$ is discrete, $X$ is continuous (Example 5.2, Activity 5.5):

$p_{Y}\left( y\right) =\int_{Set}p_{Y\vert X}\left( y\vert x\right) f_{X}\left( x\right) dx$

d) If $Y|X$ is continuous, $X$ is discrete (Activity 5.12):

$f_{Y}\left( y\right) =\sum_{Set}f_{Y\vert X}\left( y\vert x\right) p_{X}\left( x\right) .$

In all cases you need to figure out $Set$ over which to sum or integrate.

Step 2. Write out the conditional densities/probabilities with the same arguments

Step 3. Reduce the result to one of known distributions using the completeness
axiom.

### Example 5.1

Let $X$ denote the number of hurricanes which form in a given year, and let $Y$ denote the number of these which make landfall. Suppose each hurricane has a probability of $\pi$ making landfall independent of other hurricanes. Given the number of hurricanes $x$, then $Y$ can be thought of as the number of successes in $x$ independent and identically distributed Bernoulli trials. We can write this as $Y|X=x\sim Bin(x,\pi )$. Suppose we also have that $X\sim Pois(\lambda )$. Find the distribution of $Y$ (noting that $X\geq Y$ ).

### Solution

Step 1. The number of hurricanes $X$ takes values $0,1,2,...$ and is distributed as Poisson. The number of landfalls for a given $X=x$ is binomial with values $y=0,...,x$. It follows that $Set=\{x:x\ge y\}$.

Write the general formula for conditional probability:

$p_{Y}\left( y\right) =\sum_{x=y}^{\infty }p_{Y\vert X}\left( y\vert x\right) p_{X}\left( x\right) .$

Step 2. Specifying the distributions:

$p_{X}\left( x\right) =e^{-\mu }\frac{\mu ^{x}}{x!},$ where $x=0,1,2,...,$

and

$P\left( Bin\left( x,\pi \right) =y\right) =p_{Y\vert X}\left( y\vert x\right) =C_{x}^{y}\pi ^{y}\left( 1-\pi \right) ^{x-y}$ where $y\leq x.$

Step 3. Reduce the result to one of known distributions:

$p_{Y}\left( y\right) =\sum_{x=y}^{\infty }C_{x}^{y}\pi ^{y}\left( 1-\pi \right) ^{x-y}e^{-\mu }\frac{\mu ^{x}}{x!}$

(pull out of summation everything that does not depend on summation variable
$x$)

$=\frac{e^{-\mu }\mu ^{y}}{y!}\pi ^{y}\sum_{x=y}^{\infty }\frac{1}{\left( x-y\right) !}\left( \mu \left( 1-\pi \right) \right) ^{x-y}$

(replace $x-y=z$ to better see the structure)

$=\frac{e^{-\mu }\mu ^{y}}{y!}\pi ^{y}\sum_{z=0}^{\infty }\frac{1}{z!}\left( \mu \left( 1-\pi \right) \right) ^{z}$

(using the completeness axiom $\sum_{x=0}^{\infty }\frac{\mu ^{x}}{x!}=e^{\mu }$ for the Poisson variable)

$=\frac{e^{-\mu }}{y!}\left( \mu \pi \right) ^{y}e^{\mu \left( 1-\pi \right) }=\frac{e^{-\mu \pi }}{y!}\left( \mu \pi \right) ^{y}=p_{Pois(\mu \pi )}\left( y\right) .$

27
Jan 21

## AP Stats and Business Stats

Its content, organization and level justify its adoption as a textbook for introductory statistics for Econometrics in most American or European universities. The book's table of contents is somewhat standard, the innovation comes in a presentation that is crisp, concise, precise and directly relevant to the Econometrics course that will follow. I think instructors and students will appreciate the absence of unnecessary verbiage that permeates many existing textbooks.

Having read Professor Mynbaev's previous books and research articles I was not surprised with his clear writing and precision. However, I was surprised with an informal and almost conversational one-on-one style of writing which should please most students. The informality belies a careful presentation where great care has been taken to present the material in a pedagogical manner.

Carlos Martins-Filho
Professor of Economics
Boulder, USA

26
May 20

2
Mar 20

## Statistical calculator

In my book I explained how one can use Excel to do statistical simulations and replace statistical tables commonly used in statistics courses. Here I go one step further by providing a free statistical calculator that replaces the following tables from the book by Newbold et al.:

Table 1 Cumulative Distribution Function, F(z), of the Standard Normal Distribution Table

Table 2 Probability Function of the Binomial Distribution

Table 5 Individual Poisson Probabilities

Table 7a Upper Critical Values of Chi-Square Distribution with $\nu$ Degrees of Freedom

Table 8 Upper Critical Values of Student’s t Distribution with $\nu$ Degrees of Freedom

Tables 9a, 9b Upper Critical Values of the F Distribution

The calculator is just a Google sheet with statistical functions, see Picture 1:

Picture 1. Calculator using Google sheet

## How to use Calculator

1. Open an account at gmail.com, if you haven't already. Open Google Drive.
3. Find the sheet on my Google drive and copy it to your Google drive (File/Make a copy). An icon of my calculator will appear in your drive. That's not the file, it's just a link to my file. To the right of it there are three dots indicating options. One of them is "Make a copy", so use that one. The copy will be in your drive. After that you can delete the link to my file. You might want to rename "Copy of Calculator" as "Calculator".
5. When you click a cell, you can enter what you need either in the formula bar at the bottom or directly in the cell. You can also see the functions I embedded in the sheet.
6. In cell A1, for example, you can enter any legitimate formula with numbers, arithmetic signs, and Google sheet functions. Be sure to start it with =,+ or - and to press the checkmark on the right of the formula bar after you finish.
7. The cells below A1 replace the tables listed above. Beside each function there is a verbal description and further to the right - a graphical illustration (which is not in Picture 1).
8. On the tab named Regression you can calculate the slope and intercept. The sample size must be 10.
9. Keep in mind that tables for continuous distributions need two functions. For example, in case of the standard normal distribution one function allows you to go from probability (area of the left tail) to the cutting value on the horizontal axis. The other function goes from the cutting value on the horizontal axis to probability.
10. Feel free to add new sheets or functions as you may need. You will have to do this on a tablet or computer.
18
Nov 19

14
Sep 19

## Music for work and pleasure

I love all kinds of music. However, while I work I prefer a quiet music. People have different names for it: chill, chillout, lounge, smooth jazz etc. I am not good at those names, I just want something that allows me to concentrate when I want and also is a pleasure to listen when I make it louder. There is plenty of such music on Youtube.

So I tune in one of online radios, for example this is good:

When I like a melody, I select its name, right-click and then click "Search Google". Most of the time it lands me on the corresponding Youtube video. Here comes the magic: there is a nice FREE program called 4K Video Downloader. You can download the whole video or you can extract just the audio. This is how my chillout collection reached 1000. They have other good products too.

Tags:
21
Jun 19

## A singer you shouldn't miss

Usually I don't make posts like this but you should listen to this phenomenon, who happens to be my compatriot:

Dimash