The magic of the distribution function
Let 
Oftentimes, working with the distribution function is an intermediate step to obtain a density 
A series of exercises below show just how useful the distribution function is.
Exercise 1. Let 
The solution is here.
The more general case of a nonlinear transformation can also be handled:
Exercise 2. Let 
Solution. The result differs depending on whether 
Differentiation of this equation produces
(the derivative of
For an example when
Exercise 3. Suppose 
Solution. The joint density 
(converting a double integral to an iterated integral and remembering that 
Now by the Leibniz integral rule
(1)
A different method is indicated in Activity 4.11, p.207 of J. Abdey, Guide ST2133.
Exercise 4. Let 
See this post.
Exercise 5. Let 
Solution. The inequality 
For 
![=1-\left[ 1-P\left( X_1\leq x\right) \right] \left[ 1-P\left( X_2\leq x\right) \right] =1-\left( 1-F_{X_1}\left( x\right) \right) \left( 1-F_{X_2}\left( x\right) \right) .](https://s0.wp.com/latex lazyload.php?latex lazyload=%3D1-%5Cleft%5B+1-P%5Cleft%28+X_1%5Cleq+x%5Cright%29+%5Cright%5D+%5Cleft%5B+1-P%5Cleft%28+X_2%5Cleq++x%5Cright%29+%5Cright%5D+%3D1-%5Cleft%28+1-F_%7BX_1%7D%5Cleft%28+x%5Cright%29+%5Cright%29+%5Cleft%28++1-F_%7BX_2%7D%5Cleft%28+x%5Cright%29+%5Cright%29+.&bg=ffffff&fg=000&s=0&c=20201002)
or the difference 
. Find the joint density. Are variables 
independent? Find their marginal densities.
or 
?
. How much is it?
is 
.
The number 
of customers making calls simultaneously is distributed as Poisson: 
Thus the total call duration for all customers is 
for 
. We put 
. Assume that customers make their decisions about calling independently.
are identically distributed and 
are independent but not necessarily exponential and Poisson, as above) for the moment generating function of 
explaining all steps.
for your particular distributions.
?
for an arbitrary random variable 
converges in probability to a random variable 
as 
for any 
. Suppose that 
for all 
and that 
as 
converges in probability to 
.
.
.
.
The constant 
and the first task is to express the above integral as an iterated integral. This is where the domain where the density is not zero should be taken into account. There is little you can do without geometry. One example of how to do this is 
is determined by a) the extreme values of 
and b) the relationship between them. The extreme values are 0 and 1 for both 
, meaning that 
is contained in the square 
The inequality 
means that we cut out of this square the triangle below the line 
(it is really the lower triangle because if from a point on the line 
to the boundary 
The boundaries may depend on 
![\left[ y,1\right]](https://s0.wp.com/latex.php?latex=%5Cleft%5B+y%2C1%5Cright%5D+&bg=ffffff&fg=000000&s=0 "\left[ y,1\right]")
and for ![\left[ 0,x\right] .](https://s0.wp.com/latex.php?latex=%5Cleft%5B+0%2Cx%5Cright%5D+.&bg=ffffff&fg=000000&s=0 "\left[ 0,x\right] .")
). In words: the joint density decomposes into a product of individual densities.
The definition of conditional densities 
.
here can be replaced by their marginal equivalents.
takes an equivalent form using the marginal density:
depends in 
depends on 
for ![x\in[ 0,1],](https://s0.wp.com/latex.php?latex=x%5Cin%5B+0%2C1%5D%2C&bg=ffffff&fg=000000&s=0 "x\in[ 0,1],")
for ![y\in \left[ 0,1\right] .](https://s0.wp.com/latex.php?latex=y%5Cin+%5Cleft%5B+0%2C1%5Cright%5D+.&bg=ffffff&fg=000000&s=0 "y\in \left[ 0,1\right] .")
for 
for 
and 
(they sum to one). The division by denomination ($1 or $5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and $5 equal to 
and 
, resp. (they also sum to one). This is summarized in the tables
it can be a 
and $1 and the corresponding probability is 
The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:
and 
resp., and their own probabilities are 
Denote the joint probabilities 
Then we have the identities
(
equations).
because in the two-dimensional table they arise as a result of summing table entries along columns or rows and 
where 
is an i.i.d. sample from 
Using 
and 
(ST2133 p.80) we have
Then 
so standardizing 
and 
gives the same result.
(identical distribution) ![=\left[ M_{X}\left( t\right) \right] ^{n}.](https://s0.wp.com/latex.php?latex=%3D%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%5Cright%29+%5Cright%5D+%5E%7Bn%7D.&bg=ffffff&fg=000000&s=0 "=\left[ M_{X}\left( t\right) \right] ^{n}.")
then
:
and using a) we get
)![=e^{-ES_{n}t/\sigma \left( S_{n}\right) }\left[ M_{X}\left( t/\sigma \left(S_{n}\right) \right) \right] ^{n}=e^{-n\mu t/\left( \sqrt{n}\sigma \right) }\left[ M_{X}\left( t/\left( \sqrt{n}\sigma \right) \right) \right] ^{n}.](https://s0.wp.com/latex.php?latex=%3De%5E%7B-ES_%7Bn%7Dt%2F%5Csigma+%5Cleft%28+S_%7Bn%7D%5Cright%29+%7D%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%2F%5Csigma+%5Cleft%28S_%7Bn%7D%5Cright%29+%5Cright%29+%5Cright%5D+%5E%7Bn%7D%3De%5E%7B-n%5Cmu+t%2F%5Cleft%28+%5Csqrt%7Bn%7D%5Csigma+%5Cright%29+%7D%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%2F%5Cleft%28+%5Csqrt%7Bn%7D%5Csigma+%5Cright%29+%5Cright%29+%5Cright%5D+%5E%7Bn%7D.&bg=ffffff&fg=000000&s=0 "=e^{-ES_{n}t/\sigma \left( S_{n}\right) }\left[ M_{X}\left( t/\sigma \left(S_{n}\right) \right) \right] ^{n}=e^{-n\mu t/\left( \sqrt{n}\sigma \right) }\left[ M_{X}\left( t/\left( \sqrt{n}\sigma \right) \right) \right] ^{n}.")
. Therefore, we obtain![M_{z_{n}}\left( t\right) =e^{-\sqrt{\lambda }t}\left[ \exp \left( \lambda\left( e^{t/\left( n\sqrt{\lambda }\right) }-1\right) \right) \right] ^{n}=e^{-t\sqrt{\lambda }+n\lambda \left( e^{t/\left( n\sqrt{\lambda }\right)}-1\right) }.](https://s0.wp.com/latex.php?latex=M_%7Bz_%7Bn%7D%7D%5Cleft%28+t%5Cright%29+%3De%5E%7B-%5Csqrt%7B%5Clambda+%7Dt%7D%5Cleft%5B+%5Cexp+%5Cleft%28+%5Clambda%5Cleft%28+e%5E%7Bt%2F%5Cleft%28+n%5Csqrt%7B%5Clambda+%7D%5Cright%29+%7D-1%5Cright%29+%5Cright%29+%5Cright%5D+%5E%7Bn%7D%3De%5E%7B-t%5Csqrt%7B%5Clambda+%7D%2Bn%5Clambda+%5Cleft%28+e%5E%7Bt%2F%5Cleft%28+n%5Csqrt%7B%5Clambda+%7D%5Cright%29%7D-1%5Cright%29+%7D.&bg=ffffff&fg=000000&s=0 "M_{z_{n}}\left( t\right) =e^{-\sqrt{\lambda }t}\left[ \exp \left( \lambda\left( e^{t/\left( n\sqrt{\lambda }\right) }-1\right) \right) \right] ^{n}=e^{-t\sqrt{\lambda }+n\lambda \left( e^{t/\left( n\sqrt{\lambda }\right)}-1\right) }.")
some students gave 
.]
leads to
are moments of 
Differentiating this equation yields
gives the rule for finding moments from MGF: 
can also be decomposed into its Taylor series:
are called cumulants and can be found using 
. Since
and 
terms of higher order for 
small.
then by c)![K_{X}\left( t\right) =K_{a+bY}\left( t\right) =\log \left[e^{at}M_{Y}\left( bt\right) \right] =at+K_{X}\left( bt\right) .](https://s0.wp.com/latex.php?latex=K_%7BX%7D%5Cleft%28+t%5Cright%29+%3DK_%7Ba%2BbY%7D%5Cleft%28+t%5Cright%29+%3D%5Clog+%5Cleft%5Be%5E%7Bat%7DM_%7BY%7D%5Cleft%28+bt%5Cright%29+%5Cright%5D+%3Dat%2BK_%7BX%7D%5Cleft%28+bt%5Cright%29+.&bg=ffffff&fg=000000&s=0 "K_{X}\left( t\right) =K_{a+bY}\left( t\right) =\log \left[e^{at}M_{Y}\left( bt\right) \right] =at+K_{X}\left( bt\right) .")
![K_{S_{n}}\left( t\right) =\log \left[ M_{X}\left( t\right) \right]^{n}=nK_{X}\left( t\right) .](https://s0.wp.com/latex.php?latex=K_%7BS_%7Bn%7D%7D%5Cleft%28+t%5Cright%29+%3D%5Clog+%5Cleft%5B+M_%7BX%7D%5Cleft%28+t%5Cright%29+%5Cright%5D%5E%7Bn%7D%3DnK_%7BX%7D%5Cleft%28+t%5Cright%29+.&bg=ffffff&fg=000000&s=0 "K_{S_{n}}\left( t\right) =\log \left[ M_{X}\left( t\right) \right]^{n}=nK_{X}\left( t\right) .")
and then IV) we have
to the series for 
from 
.]
for each 
(ST2133 example 3.42) and hence for the standard normal
and convergence in distribution of 
) Let 
converges for each 
, then 
converges in distribution to 
converges in distribution to the standard normal, which is the central limit theorem.
is a sufficient statistic.
the joint density is
is a sufficient statistic for 
stating all necessary theoretical facts.
is minimal sufficient if and only if level sets of 
coincide with sets on which the ratio 
does not depend on 
![f_{X}\left( x,\lambda \right) /f_{X}\left( y,\lambda \right) =\frac{\lambda^{\Sigma x_{i}}e^{-n\lambda }}{\Pi _{i=1}^{n}x_{i}!}\left[ \frac{\lambda^{\Sigma y_{i}}e^{-n\lambda }}{\Pi _{i=1}^{n}y_{i}!}\right] ^{-1}](https://s0.wp.com/latex.php?latex=f_%7BX%7D%5Cleft%28+x%2C%5Clambda+%5Cright%29+%2Ff_%7BX%7D%5Cleft%28+y%2C%5Clambda+%5Cright%29+%3D%5Cfrac%7B%5Clambda%5E%7B%5CSigma+x_%7Bi%7D%7De%5E%7B-n%5Clambda+%7D%7D%7B%5CPi+_%7Bi%3D1%7D%5E%7Bn%7Dx_%7Bi%7D%21%7D%5Cleft%5B+%5Cfrac%7B%5Clambda%5E%7B%5CSigma+y_%7Bi%7D%7De%5E%7B-n%5Clambda+%7D%7D%7B%5CPi+_%7Bi%3D1%7D%5E%7Bn%7Dy_%7Bi%7D%21%7D%5Cright%5D+%5E%7B-1%7D&bg=ffffff&fg=000000&s=0 "f_{X}\left( x,\lambda \right) /f_{X}\left( y,\lambda \right) =\frac{\lambda^{\Sigma x_{i}}e^{-n\lambda }}{\Pi _{i=1}^{n}x_{i}!}\left[ \frac{\lambda^{\Sigma y_{i}}e^{-n\lambda }}{\Pi _{i=1}^{n}y_{i}!}\right] ^{-1}")
![=\lambda ^{\left[ \Sigma x_{i}-\Sigma y_{i}\right] }\frac{\Pi _{i=1}^{n}y_{i}!}{\Pi_{i=1}^{n}x_{i}!}.](https://s0.wp.com/latex.php?latex=%3D%5Clambda+%5E%7B%5Cleft%5B+%5CSigma+x_%7Bi%7D-%5CSigma+y_%7Bi%7D%5Cright%5D+%7D%5Cfrac%7B%5CPi+_%7Bi%3D1%7D%5E%7Bn%7Dy_%7Bi%7D%21%7D%7B%5CPi_%7Bi%3D1%7D%5E%7Bn%7Dx_%7Bi%7D%21%7D.&bg=ffffff&fg=000000&s=0 "=\lambda ^{\left[ \Sigma x_{i}-\Sigma y_{i}\right] }\frac{\Pi _{i=1}^{n}y_{i}!}{\Pi_{i=1}^{n}x_{i}!}.")
The last condition describes level sets of 
Thus it is minimal sufficient.
) by its sample analog (
[Try to do this for the 
for a random sample.
for linear regression allows one to do without long calculations contained in the guide ST 2134 by J. Abdey.
be independent observations from 
. 1) 
is distributed as 
2) The estimators 
and 
4) 
5) 
converges in distribution to 
where 
is distributed as 
Putting 
and 
(a vector of ones) we 
we have 
Further,
(see Guide ST2133, 2021, p.88). For the standard normal we get
(same guide, p.84) we see that
![\left(\frac{\sigma ^{2}}{n-1}\right) ^{2}\left[ Var\left( z_{1}^{2}\right)+...+Var\left( z_{n-1}^{2}\right) \right] =\frac{\sigma ^{4}}{\left(n-1\right) ^{2}}2\left( n-1\right) =\frac{2\sigma ^{4}}{n-1}.](https://s0.wp.com/latex.php?latex=%5Cleft%28%5Cfrac%7B%5Csigma+%5E%7B2%7D%7D%7Bn-1%7D%5Cright%29+%5E%7B2%7D%5Cleft%5B+Var%5Cleft%28+z_%7B1%7D%5E%7B2%7D%5Cright%29%2B...%2BVar%5Cleft%28+z_%7Bn-1%7D%5E%7B2%7D%5Cright%29+%5Cright%5D+%3D%5Cfrac%7B%5Csigma+%5E%7B4%7D%7D%7B%5Cleft%28n-1%5Cright%29+%5E%7B2%7D%7D2%5Cleft%28+n-1%5Cright%29+%3D%5Cfrac%7B2%5Csigma+%5E%7B4%7D%7D%7Bn-1%7D.&bg=ffffff&fg=000000&s=0 "\left(\frac{\sigma ^{2}}{n-1}\right) ^{2}\left[ Var\left( z_{1}^{2}\right)+...+Var\left( z_{n-1}^{2}\right) \right] =\frac{\sigma ^{4}}{\left(n-1\right) ^{2}}2\left( n-1\right) =\frac{2\sigma ^{4}}{n-1}.")
and this converges in distribution to 
by the central limit theorem.
for a statistic, where 
or 
if the distribution of 
be the joint density of the vector 
, where 
fixed at the observed data. The estimator depends on the data and can be denoted 
and 
because 
depends on 
A sufficient statistic is all you need to find the ML estimator. This interpretation is easier to understand than the definition of sufficiency.
there exists a function 
for a constant 
(which in general can be a constant vector). See the 
When 
generally contains more than one point, this shows that the level sets of 
Since this is true for any 
such that
Maximizing 
we get 
Since 
does not depend on 
(2) tells us that
means that 
Assuming that 
, on Chart 1 we see that 
Hence, using 
for all 
By the 
and 
answer questions similar to 1 and 2.
Just applying (2) to 
and 
we get
the 
with independent standard normals 
we can write 
where the chi-squared variables on the right are independent and all have one degree of freedom. This is because deterministic (here quadratic) functions of independent variables are independent.
Then by the 
is a two-parametric family of densities. For 
the density is defined by
(replace 
)
and need to determine which gamma density this is. The power of the exponent gives you 
and the power of 
It follows that the normalizing constant should be 
and the density is 
is closed under convolutions. Because of the associativity property 
it is enough to prove this for the case of two gamma densities.
![=\int_{0}^{x}\left[ \frac{1}{\Gamma \left( \mu \right) }\alpha ^{\mu }\left( x-y\right) ^{\mu -1}e^{-\alpha \left( x-y\right) }\right] \left[ \frac{1}{\Gamma \left( \nu \right) }\alpha ^{\nu }y^{\nu -1}e^{-\alpha y} \right] dy](https://s0.wp.com/latex.php?latex=%3D%5Cint_%7B0%7D%5E%7Bx%7D%5Cleft%5B+%5Cfrac%7B1%7D%7B%5CGamma+%5Cleft%28+%5Cmu+%5Cright%29+%7D%5Calpha+%5E%7B%5Cmu++%7D%5Cleft%28+x-y%5Cright%29+%5E%7B%5Cmu+-1%7De%5E%7B-%5Calpha+%5Cleft%28+x-y%5Cright%29+%7D%5Cright%5D+%5Cleft%5B++%5Cfrac%7B1%7D%7B%5CGamma+%5Cleft%28+%5Cnu+%5Cright%29+%7D%5Calpha+%5E%7B%5Cnu+%7Dy%5E%7B%5Cnu+-1%7De%5E%7B-%5Calpha+y%7D++%5Cright%5D+dy&bg=ffffff&fg=000000&s=0 "=\int_{0}^{x}\left[ \frac{1}{\Gamma \left( \mu \right) }\alpha ^{\mu }\left( x-y\right) ^{\mu -1}e^{-\alpha \left( x-y\right) }\right] \left[ \frac{1}{\Gamma \left( \nu \right) }\alpha ^{\nu }y^{\nu -1}e^{-\alpha y} \right] dy")
)
shows that this sum is again a beta distribution with the same 
is smooth on 
so its integrability is determined by its behavior at 
and 
. Because of the exponent, it is integrable in the neighborhood of 
The singularity at 
In all calculations involving the gamma function one should remember that its argument should be positive.
if 
because 
of the standard normal 
we see that
)
)
You must be logged in to post a comment.