Solution to problem 1(b) from exam ST2133 ZA, 2019
Simple tools for combinatorial problems
Before solving the problem, it is useful to compare the case of independent events with that of dependent events.
Suppose the events 
(1)
Now assume that the event 
Similarly, by multiplying and dividing many times, we get
(2)
Equation (2) is called a chain rule for probability. Several of my students have been able to solve the problem without explicitly using (2). It is advisable to use (2) or other relevant theoretical properties to achieve clarity and avoid errors.
Problem statement and solution
Suppose there are 
Part i.
Suppose one ball is drawn at a time at random with replacement from the bag. Let 
Solution. Most students answer that this is a hypergeometric distribution with probabilities given by 
Part ii.
Now suppose one ball is drawn at random at a time without replacement from the bag. Let 
Solution. Let us denote 
Obviously,
Next, using (2)
Further,
Finally,
The results can be summarized in a table:
This distribution is not of one of standard types.
Part iii.
Suppose two balls are drawn at the same time at random with replacement from the bag. Let 
Hence, show that the probability mass function for
Solution. Using the same notation as before,
All other events (two red balls or one green and one red) are considered a failure. Thus we have a hypergeometric distribution with
and
![q^{z-1}p=\left[ \frac{n^{2}+5n}{\left( n+2\right) \left( n+3\right) }\right]^{z-1}\frac{6}{\left( n+2\right) \left( n+3\right) }=\frac{6\left( n^{2}+5n\right) ^{z-1}}{\left( n+2\right) ^{z}\left( n+3\right) ^{2}},](https://s0.wp.com/latex.php?latex=q%5E%7Bz-1%7Dp%3D%5Cleft%5B+%5Cfrac%7Bn%5E%7B2%7D%2B5n%7D%7B%5Cleft%28+n%2B2%5Cright%29+%5Cleft%28+n%2B3%5Cright%29+%7D%5Cright%5D%5E%7Bz-1%7D%5Cfrac%7B6%7D%7B%5Cleft%28+n%2B2%5Cright%29+%5Cleft%28+n%2B3%5Cright%29+%7D%3D%5Cfrac%7B6%5Cleft%28++n%5E%7B2%7D%2B5n%5Cright%29+%5E%7Bz-1%7D%7D%7B%5Cleft%28+n%2B2%5Cright%29+%5E%7Bz%7D%5Cleft%28+n%2B3%5Cright%29+%5E%7B2%7D%7D%2C&bg=ffffff&fg=000&s=0&c=20201002)
where 
runs over real numbers, is called a distribution function of 
In statistics, many formulas are derived with the help of 
The motivation and properties are given 
using the link
that is, 
where 
and 
Find the link between 
and 
Find the link between 
where 
is a deterministic function. Suppose that 
exists. Find the link between 
is equivalent to 
Also for simplicity suppose that 
for any 
Then
where 
and 
What are the distribution function and density of 
equals 
so
is zero on the left half-axis)
be two independent random variables with densities 
and 
and 
holds if and only if both 
and 
hold. This means that the event 
coincides with the event 
It follows by independence that
we need one more trick, namely, pass to the complementary event by writing
coincides with the event 
Hence, by independence
![=1-\left[ 1-P\left( X_{1}\leq x\right) \right] \left[ 1-P\left( X_{2}\leq x\right) \right] =1-\left( 1-F_{X_{1}}\left( x\right) \right) \left(1-F_{X_{2}}\left( x\right) \right) .](https://s0.wp.com/latex.php?latex=%3D1-%5Cleft%5B+1-P%5Cleft%28+X_%7B1%7D%5Cleq+x%5Cright%29+%5Cright%5D+%5Cleft%5B+1-P%5Cleft%28+X_%7B2%7D%5Cleq++x%5Cright%29+%5Cright%5D+%3D1-%5Cleft%28+1-F_%7BX_%7B1%7D%7D%5Cleft%28+x%5Cright%29+%5Cright%29+%5Cleft%281-F_%7BX_%7B2%7D%7D%5Cleft%28+x%5Cright%29+%5Cright%29+.&bg=ffffff&fg=000&s=0&c=20201002)
or the difference 
. Find the joint density. Are variables 
independent? Find their marginal densities.
or 
?
. How much is it?
is 
.
The number 
of customers making calls simultaneously is distributed as Poisson: 
Thus the total call duration for all customers is 
for 
. We put 
. Assume that customers make their decisions about calling independently.
are identically distributed and 
are independent but not necessarily exponential and Poisson, as above) for the moment generating function of 
explaining all steps.
for your particular distributions.
?
for an arbitrary random variable 
converges in probability to a random variable 
as 
for any 
. Suppose that 
for all 
and that 
as 
converges in probability to 
.
.
.
.
The constant 
and the first task is to express the above integral as an iterated integral. This is where the domain where the density is not zero should be taken into account. There is little you can do without geometry. One example of how to do this is 
is determined by a) the extreme values of 
and b) the relationship between them. The extreme values are 0 and 1 for both 
, meaning that 
is contained in the square 
The inequality 
means that we cut out of this square the triangle below the line 
(it is really the lower triangle because if from a point on the line 
to the boundary 
The boundaries may depend on 
![\left[ y,1\right]](https://s0.wp.com/latex.php?latex=%5Cleft%5B+y%2C1%5Cright%5D+&bg=ffffff&fg=000&s=0&c=20201002)
and for ![\left[ 0,x\right] .](https://s0.wp.com/latex.php?latex=%5Cleft%5B+0%2Cx%5Cright%5D+.&bg=ffffff&fg=000&s=0&c=20201002)
(this is a direct analog of the independence condition for events 
). In words: the joint density decomposes into a product of individual densities.
The definition of conditional densities 
.
here can be replaced by their marginal equivalents.
takes an equivalent form using the marginal density:
depends in 
depends on 
for ![x\in[ 0,1],](https://s0.wp.com/latex.php?latex=x%5Cin%5B+0%2C1%5D%2C&bg=ffffff&fg=000&s=0&c=20201002)
for ![y\in \left[ 0,1\right] .](https://s0.wp.com/latex.php?latex=y%5Cin+%5Cleft%5B+0%2C1%5Cright%5D+.&bg=ffffff&fg=000&s=0&c=20201002)
for 
for 
and 
(they sum to one). The division by denomination ($1 or $5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and $5 equal to 
and 
, resp. (they also sum to one). This is summarized in the tables
it can be a 
and $1 and the corresponding probability is 
The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:
and 
resp., and their own probabilities are 
Denote the joint probabilities 
Then we have the identities
(
equations).
because in the two-dimensional table they arise as a result of summing table entries along columns or rows and 
where 
is an i.i.d. sample from 
Using 
and 
(ST2133 p.80) we have
Then 
so standardizing 
and 
gives the same result.
(identical distribution) ![=\left[ M_{X}\left( t\right) \right] ^{n}.](https://s0.wp.com/latex.php?latex=%3D%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%5Cright%29+%5Cright%5D+%5E%7Bn%7D.&bg=ffffff&fg=000&s=0&c=20201002)
then
:
and using a) we get
)![=e^{-ES_{n}t/\sigma \left( S_{n}\right) }\left[ M_{X}\left( t/\sigma \left( S_{n}\right) \right) \right] ^{n}=e^{-n\mu t/\left( \sqrt{n}\sigma \right) }% \left[ M_{X}\left( t/\left( \sqrt{n}\sigma \right) \right) \right] ^{n}.](https://s0.wp.com/latex.php?latex=%3De%5E%7B-ES_%7Bn%7Dt%2F%5Csigma+%5Cleft%28+S_%7Bn%7D%5Cright%29+%7D%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%2F%5Csigma+%5Cleft%28++S_%7Bn%7D%5Cright%29+%5Cright%29+%5Cright%5D+%5E%7Bn%7D%3De%5E%7B-n%5Cmu+t%2F%5Cleft%28+%5Csqrt%7Bn%7D%5Csigma+%5Cright%29+%7D%25++%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%2F%5Cleft%28+%5Csqrt%7Bn%7D%5Csigma+%5Cright%29+%5Cright%29+%5Cright%5D+%5E%7Bn%7D.&bg=ffffff&fg=000&s=0&c=20201002)
. Therefore, we obtain![M_{z_{n}}\left( t\right) =e^{-\sqrt{\lambda }t}\left[ \exp \left( \lambda \left( e^{t/\left( n\sqrt{\lambda }\right) }-1\right) \right) \right] ^{n}= e^{-t\sqrt{\lambda }+n\lambda \left( e^{t/\left( n\sqrt{\lambda }\right) }-1\right) }.](https://s0.wp.com/latex.php?latex=M_%7Bz_%7Bn%7D%7D%5Cleft%28+t%5Cright%29+%3De%5E%7B-%5Csqrt%7B%5Clambda+%7Dt%7D%5Cleft%5B+%5Cexp+%5Cleft%28+%5Clambda++%5Cleft%28+e%5E%7Bt%2F%5Cleft%28+n%5Csqrt%7B%5Clambda+%7D%5Cright%29+%7D-1%5Cright%29+%5Cright%29+%5Cright%5D+%5E%7Bn%7D%3D++e%5E%7B-t%5Csqrt%7B%5Clambda+%7D%2Bn%5Clambda+%5Cleft%28+e%5E%7Bt%2F%5Cleft%28+n%5Csqrt%7B%5Clambda+%7D%5Cright%29++%7D-1%5Cright%29+%7D.&bg=ffffff&fg=000&s=0&c=20201002)
some students gave 
.]
leads to
are moments of 
Differentiating this equation yields
gives the rule for finding moments from MGF: 
can also be decomposed into its Taylor series:
are called cumulants and can be found using 
. Since
and 
terms of higher order for 
small.
then by c)![K_{X}\left( t\right) =K_{a+bY}\left( t\right) =\log \left[ e^{at}M_{Y}\left( bt\right) \right] =at+K_{X}\left( bt\right) .](https://s0.wp.com/latex.php?latex=K_%7BX%7D%5Cleft%28+t%5Cright%29+%3DK_%7Ba%2BbY%7D%5Cleft%28+t%5Cright%29+%3D%5Clog+%5Cleft%5B++e%5E%7Bat%7DM_%7BY%7D%5Cleft%28+bt%5Cright%29+%5Cright%5D+%3Dat%2BK_%7BX%7D%5Cleft%28+bt%5Cright%29+.&bg=ffffff&fg=000&s=0&c=20201002)
![K_{S_{n}}\left( t\right) =\log \left[ M_{X}\left( t\right) \right] ^{n}=nK_{X}\left( t\right) .](https://s0.wp.com/latex.php?latex=K_%7BS_%7Bn%7D%7D%5Cleft%28+t%5Cright%29+%3D%5Clog+%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%5Cright%29+%5Cright%5D++%5E%7Bn%7D%3DnK_%7BX%7D%5Cleft%28+t%5Cright%29+.&bg=ffffff&fg=000&s=0&c=20201002)
and then IV) we have
to the series for 
from 
.]
for each 
(ST2133 example 3.42) and hence for the standard normal
and convergence in distribution of 
) Let 
converges for each 
, then 
converges in distribution to 
converges in distribution to the standard normal, which is the central limit theorem.
is a sufficient statistic.
the joint density is
is a sufficient statistic for 
stating all necessary theoretical facts.
is minimal sufficient if and only if level sets of 
coincide with sets on which the ratio 
does not depend on 
![f_{X}\left( x,\lambda \right) /f_{X}\left( y,\lambda \right) =\frac{\lambda ^{\Sigma x_{i}}e^{-n\lambda }}{\Pi _{i=1}^{n}x_{i}!}\left[ \frac{\lambda ^{\Sigma y_{i}}e^{-n\lambda }}{\Pi _{i=1}^{n}y_{i}!}\right] ^{-1}=\lambda ^{ \left[ \Sigma x_{i}-\Sigma y_{i}\right] }\frac{\Pi _{i=1}^{n}y_{i}!}{\Pi _{i=1}^{n}x_{i}!}.](https://s0.wp.com/latex.php?latex=f_%7BX%7D%5Cleft%28+x%2C%5Clambda+%5Cright%29+%2Ff_%7BX%7D%5Cleft%28+y%2C%5Clambda+%5Cright%29+%3D%5Cfrac%7B%5Clambda++%5E%7B%5CSigma+x_%7Bi%7D%7De%5E%7B-n%5Clambda+%7D%7D%7B%5CPi+_%7Bi%3D1%7D%5E%7Bn%7Dx_%7Bi%7D%21%7D%5Cleft%5B+%5Cfrac%7B%5Clambda++%5E%7B%5CSigma+y_%7Bi%7D%7De%5E%7B-n%5Clambda+%7D%7D%7B%5CPi+_%7Bi%3D1%7D%5E%7Bn%7Dy_%7Bi%7D%21%7D%5Cright%5D+%5E%7B-1%7D%3D%5Clambda+%5E%7B++%5Cleft%5B+%5CSigma+x_%7Bi%7D-%5CSigma+y_%7Bi%7D%5Cright%5D+%7D%5Cfrac%7B%5CPi+_%7Bi%3D1%7D%5E%7Bn%7Dy_%7Bi%7D%21%7D%7B%5CPi++_%7Bi%3D1%7D%5E%7Bn%7Dx_%7Bi%7D%21%7D.&bg=ffffff&fg=000&s=0&c=20201002)
The last condition describes level sets of 
Thus it is minimal sufficient.
) by its sample analog (
[Try to do this for the 
for a random sample.
for linear regression allows one to do without long calculations contained in the guide ST 2134 by J. Abdey.
be independent observations from 
. 1) 
is distributed as 
2) The estimators 
and 
4) 
5) 
converges in distribution to 
where 
is distributed as 
Putting 
and 
(a vector of ones) we 
we have 
Further,
(see Guide ST2133, 2021, p.88). For the standard normal we get
(same guide, p.84) we see that
![\left(\frac{\sigma ^{2}}{n-1}\right) ^{2}\left[ Var\left( z_{1}^{2}\right) +...+Var\left( z_{n-1}^{2}\right) \right] =\frac{\sigma ^{4}}{\left( n-1\right) ^{2}}2\left( n-1\right) =\frac{2\sigma ^{4}}{n-1}.](https://s0.wp.com/latex.php?latex=%5Cleft%28%5Cfrac%7B%5Csigma+%5E%7B2%7D%7D%7Bn-1%7D%5Cright%29+%5E%7B2%7D%5Cleft%5B+Var%5Cleft%28+z_%7B1%7D%5E%7B2%7D%5Cright%29++%2B...%2BVar%5Cleft%28+z_%7Bn-1%7D%5E%7B2%7D%5Cright%29+%5Cright%5D+%3D%5Cfrac%7B%5Csigma+%5E%7B4%7D%7D%7B%5Cleft%28++n-1%5Cright%29+%5E%7B2%7D%7D2%5Cleft%28+n-1%5Cright%29+%3D%5Cfrac%7B2%5Csigma+%5E%7B4%7D%7D%7Bn-1%7D.&bg=ffffff&fg=000&s=0&c=20201002)
and this converges in distribution to 
by the central limit theorem.
for a statistic, where 
or 
if the distribution of 
be the joint density of the vector 
, where 
fixed at the observed data. The estimator depends on the data and can be denoted 
and 
because 
depends on 
A sufficient statistic is all you need to find the ML estimator. This interpretation is easier to understand than the definition of sufficiency.
there exists a function 
for a constant 
(which in general can be a constant vector). See the 
When 
generally contains more than one point, this shows that the level sets of 
Since this is true for any 
such that
Maximizing 
we get 
Since 
does not depend on 
(2) tells us that
means that 
Assuming that 
, on Chart 1 we see that 
Hence, using 
for all 
By the 
and 
answer questions similar to 1 and 2.
Just applying (2) to 
and 
we get
the 
with independent standard normals 
we can write 
where the chi-squared variables on the right are independent and all have one degree of freedom. This is because deterministic (here quadratic) functions of independent variables are independent.
Then by the 
is a two-parametric family of densities. For 
the density is defined by
(replace 
)
and need to determine which gamma density this is. The power of the exponent gives you 
and the power of 
It follows that the normalizing constant should be 
and the density is 
is closed under convolutions. Because of the associativity property 
it is enough to prove this for the case of two gamma densities.
![=\int_{0}^{x}\left[ \frac{1}{\Gamma \left( \mu \right) }\alpha ^{\mu}\left( x-y\right) ^{\mu -1}e^{-\alpha \left( x-y\right) }\right] \left[\frac{1}{\Gamma \left( \nu \right) }\alpha ^{\nu }y^{\nu -1}e^{-\alpha y}\right] dy](https://s0.wp.com/latex.php?latex=%3D%5Cint_%7B0%7D%5E%7Bx%7D%5Cleft%5B+%5Cfrac%7B1%7D%7B%5CGamma+%5Cleft%28+%5Cmu+%5Cright%29+%7D%5Calpha+%5E%7B%5Cmu%7D%5Cleft%28+x-y%5Cright%29+%5E%7B%5Cmu+-1%7De%5E%7B-%5Calpha+%5Cleft%28+x-y%5Cright%29+%7D%5Cright%5D+%5Cleft%5B%5Cfrac%7B1%7D%7B%5CGamma+%5Cleft%28+%5Cnu+%5Cright%29+%7D%5Calpha+%5E%7B%5Cnu+%7Dy%5E%7B%5Cnu+-1%7De%5E%7B-%5Calpha+y%7D%5Cright%5D+dy&bg=ffffff&fg=000&s=0&c=20201002)
)
shows that this sum is again a beta distribution with the same
You must be logged in to post a comment.