Raising the bar

Solution to Question 1 from UoL exam 2020

The assessment was an open-book take-home online assessment with a 24-hour window. No attempt was made to prevent cheating, except a warning, which was pretty realistic. Before an exam it's a good idea to see my checklist.

Question 1. Consider the following ARMA(1,1) process:

(1) $latex z_{t}=\gamma +\alpha z_{t-1}+\varepsilon _{t}+\theta \varepsilon _{t-1}$

where $latex \varepsilon_{t}$ is a zero-mean white noise process with variance $latex \sigma ^{2}$, and assume $latex |\alpha |,|\theta |<1$ and $latex \alpha+\theta \neq 0$, which together make sure $latex z_{t}$ is covariance stationary.

(a) [20 marks] Calculate the conditional and unconditional means of $latex z_{t}$, that is, $latex E_{t-1}[z_{t}]$ and $latex E[z_{t}].$

(b) [20 marks] Set $latex \alpha =0$. Derive the autocovariance and autocorrelation function of this process for all lags as functions of the parameters $latex \theta $ and $latex \sigma $.

(c) [30 marks] Assume now $latex \alpha \neq 0$. Calculate the conditional and unconditional variances of $latex z_{t},$ that is, $latex Var_{t-1}[z_{t}]$ and $latex Var[z_{t}].$

Hint: for the unconditional variance, you might want to start by deriving the unconditional covariance between the variable and the innovation term, i.e., $latex Cov[z_{t},\varepsilon _{t}].$

(d) [30 marks] Derive the autocovariance and autocorrelation for lags of 1 and 2 as functions of the parameters of the model.

Hint: use the hint of part (c).

Solution

Part (a)

Reminder: The definition of a zero-mean white noise process is

(2) $latex E\varepsilon _t=0,\ Var(\varepsilon_t)=E\varepsilon_t^2=\sigma^2$ for all $latex t$ and $latex Cov(\varepsilon_j,\varepsilon_i)=E\varepsilon_j\varepsilon_i=0$ for all $latex i\neq j.$

A variable indexed $latex t-1$ is known at moment $latex t-1$ and at all later moments and behaves like a constant for conditioning at such moments.

Moment $latex t$ is future relative to $latex t-1.$  The future is unpredictable and the best guess about the future error is zero.

The recurrent relationship in (1) shows that

(3) $latex z_{t-1}=\gamma +\alpha z_{t-2}+...$ does not depend on the information that arrives at time $latex t$ and later.

Hence, using also linearity of conditional means,

(4) $latex E_{t-1}z_t=E_{t-1}\gamma+\alpha E_{t-1}z_{t-1}+E_{t-1}\varepsilon_t+\theta E_{t-1}\varepsilon_{t-1}=\gamma+\alpha z_{t-1}+\theta\varepsilon_{t-1}.$

The law of iterated expectations (LIE): application of $latex E_{t-1},$ based on information available at time $latex t-1,$ and subsequent application of $latex E,$ based on no information, gives the same result as application of $latex E.$

$latex Ez_t=E[E_{t-1}z_{t}]=E\gamma+\alpha Ez_{t-1}+\theta E\varepsilon_{t-1}=\gamma+\alpha Ez_{t-1}.$

Since $latex z_{t}$ is covariance stationary, its means across times are the same, so $latex Ez_{t}=\gamma +\alpha Ez_{t}$ and $latex Ez_{t}=\frac{\gamma }{1-\alpha }.$

Part (b)

With $latex \alpha =0$ we get $latex z_{t}=\gamma+\varepsilon_{t}+\theta\varepsilon_{t-1}$ and from part (a) $latex Ez_{t}=\gamma .$ Using (2), we find variance

$latex Var(z_{t})=E(z_{t}-Ez_{t})^{2}=E(\varepsilon_{t}^{2}+2\theta\varepsilon_{t}\varepsilon_{t-1}+\theta ^{2}\varepsilon_{t-2}^{2})=(1+\theta^{2})\sigma^{2}$

and first autocovariance

(5) $latex \gamma_{1}=Cov(z_{t},z_{t-1})=E(z_{t}-Ez_{t})(z_{t-1}-Ez_{t-1})$

$latex =E(\varepsilon_{t}+\theta\varepsilon_{t-1})(\varepsilon_{t-1}+\theta\varepsilon_{t-2})=\theta E\varepsilon _{t-1}^{2}=\theta\sigma^{2}.$

Second and higher autocovariances are zero because the subscripts of epsilons don't overlap.

Autocorrelation function: $latex \rho_{0}=\frac{Cov(z_{t},z_{t})}{\sqrt{Var(z_{t})Var(z_{t})}}=1$ (this is always true),

$latex \rho_{1}=\frac{Cov(z_{t},z_{t-1})}{\sqrt{Var(z_{t})Var(z_{t-1})}}=\frac{\theta\sigma ^{2}}{(1+\theta^{2})\sigma^{2}}=\frac{\theta}{1+\theta^{2}},$ $latex \rho_{j}=0$ for $latex j>1.$

This is characteristic of MA processes: their autocorrelations are zero starting from some point.

Part (c)

If we replace all expectations in the definition of variance, we obtain the definition of conditional variance. From (1) and (4)

$latex Var_{t-1}(z_{t})=E_{t-1}(z_{t}-E_{t-1}z_{t})^{2}=E_{t-1}\varepsilon_{t}^{2}=\sigma ^{2}.$

By the law of total variance

(6) $latex Var(z_{t})=EVar_{t-1}(z_{t})+Var(E_{t-1}z_{t})=\sigma ^{2}+Var(\gamma+\alpha z_{t-1}+\theta \varepsilon _{t-1})=$

(an additive constant does not affect variance)

$latex =\sigma^{2}+Var(\alpha z_{t-1}+\theta\varepsilon_{t-1})=\sigma^{2}+\alpha^{2}Var(z_{t})+2\alpha\theta Cov(z_{t-1},\varepsilon_{t-1})+\theta ^{2}Var(\varepsilon_{t-1}).$

By the LIE and (3)

$latex Cov(z_{t-1},\varepsilon_{t-1})=Cov(\gamma+\alpha z_{t-2}+\varepsilon_{t-1}+\theta \varepsilon_{t-2},\varepsilon_{t-1})$

$latex =\alpha Cov(z_{t-2},\varepsilon_{t-1})+E\varepsilon_{t-1}^{2}+\theta EE_{t-2}\varepsilon_{t-2}\varepsilon_{t-1}=\sigma^{2}+\theta E(\varepsilon_{t-2}E_{t-2}\varepsilon_{t-1}).$

Here $latex E_{t-2}\varepsilon _{t-1}=0,$ so

(7) $latex Cov(z_{t-1},\varepsilon_{t-1})=\sigma^{2}.$

This equation leads to

$latex Var(z_{t})=Var(\gamma+\alpha z_{t-1}+\varepsilon_{t}+\theta \varepsilon_{t-1})=\alpha^{2}Var(z_{t-1})+Var(\varepsilon_{t})+\theta^{2}Var(\varepsilon_{t-1})+$

$latex +2\alpha Cov(z_{t-1},\varepsilon_{t})+2\alpha\theta Cov(z_{t-1},\varepsilon_{t-1})+2\theta Cov(\varepsilon_{t},\varepsilon_{t-1})=\alpha ^{2}Var(z_{t})+\sigma ^{2}+\theta^{2}\sigma^{2}+2\alpha\theta\sigma^{2}$

and, finally,

(8) $latex Var(z_{t})=\frac{(1+2\alpha\theta+\theta^{2})\sigma^{2}}{1-\alpha^{2}}.$

Part (d)

From (7)

(9) $latex Cov(z_{t-1},\varepsilon_{t-2})=Cov(\gamma+\alpha z_{t-2}+\varepsilon_{t-1}+\theta\varepsilon_{t-2},\varepsilon_{t-2})$

$latex =\alpha Cov(z_{t-2},\varepsilon_{t-2})+\theta Var(\varepsilon_{t-2})=(\alpha+\theta)\sigma^{2}.$

It follows that

$latex Cov(z_{t},z_{t-1})=Cov(\gamma+\alpha z_{t-1}+\varepsilon_{t}+\theta\varepsilon_{t-1},\gamma+\alpha z_{t-2}+\varepsilon_{t-1}+\theta
\varepsilon_{t-2})=$

(a constant is not correlated with anything)

$latex =\alpha ^{2}Cov(z_{t-1},z_{t-2})+\alpha Cov(z_{t-1},\varepsilon_{t-1})+\alpha \theta Cov(z_{t-1},\varepsilon_{t-2})+$

$latex +\alpha Cov(\varepsilon_{t},z_{t-2})+Cov(\varepsilon_{t},\varepsilon_{t-1})+\theta Cov(\varepsilon_{t},\varepsilon_{t-2})+$

$latex +\theta\alpha Cov(\varepsilon_{t-1},z_{t-2})+\theta Var(\varepsilon_{t-1})+\theta ^{2}Cov(\varepsilon_{t-1},\varepsilon_{t-2}).$

From (7) $latex Cov(z_{t-2},\varepsilon_{t-2})=\sigma ^{2}$ and from (9) $latex Cov(z_{t-1},\varepsilon_{t-2})=(\alpha+\theta)\sigma^{2}.$

From (3) $latex Cov(\varepsilon_{t},z_{t-2})=Cov(\varepsilon_{t-1},z_{t-2})=0.$

Using also the white noise properties and stationarity of $latex z_{t}$

$latex Cov(z_{t},z_{t-1})=Cov(z_{t-1},z_{t-2})=\gamma_{1},$

we are left with

$latex \gamma _{1}=\alpha ^{2}\gamma _{1}+\alpha \sigma
^{2}+\alpha \theta (\alpha +\theta )\sigma ^{2}+\theta \sigma ^{2}=\alpha
^{2}\gamma _{1}+(1+\alpha \theta )(\alpha +\theta )\sigma ^{2}.$

Hence,

$latex \gamma _{1}=\frac{(1+\alpha \theta )(\alpha +\theta )\sigma ^{2}}{1-\alpha
^{2}}$

and using (8)

$latex \rho _{0}=1,$ $latex \rho _{1}=\frac{(1+\alpha \theta )(\alpha +\theta )}{
1+2\alpha \theta +\theta ^{2}}.$

The finish is close.

$latex Cov(z_{t},z_{t-2})=Cov(\gamma+\alpha z_{t-1}+\varepsilon_{t}+\theta\varepsilon_{t-1},\gamma+\alpha z_{t-3}+\varepsilon_{t-2}+\theta
\varepsilon _{t-3})=$

$latex =\alpha ^{2}Cov(z_{t-1},z_{t-3})+\alpha Cov(z_{t-1},\varepsilon_{t-2})+\alpha\theta Cov(z_{t-1},\varepsilon_{t-3})+$

$latex +\alpha Cov(\varepsilon_{t},z_{t-3})+Cov(\varepsilon_{t},\varepsilon_{t-2})+\theta Cov(\varepsilon_{t},\varepsilon_{t-3})+$

$latex +\theta \alpha Cov(\varepsilon_{t-1},z_{t-3})+\theta Cov(\varepsilon_{t-1},\varepsilon_{t-2})+\theta ^{2}Cov(\varepsilon_{t-1},\varepsilon_{t-3}).$

This simplifies to

(10) $latex Cov(z_{t},z_{t-2})=\alpha ^{2}Cov(z_{t-1},z_{t-3})+\alpha (\alpha
+\theta )\sigma ^{2}+\alpha \theta Cov(z_{t-1},\varepsilon _{t-3}).$

By (7)

$latex Cov(z_{t-1},\varepsilon_{t-3})=Cov(\gamma+\alpha z_{t-2}+\varepsilon_{t-1}+\theta \varepsilon_{t-2},\varepsilon_{t-3})=\alpha Cov(z_{t-2},\varepsilon_{t-3})=$

$latex =\alpha Cov(\gamma+\alpha z_{t-3}+\varepsilon_{t-2}+\theta \varepsilon_{t-3},\varepsilon_{t-3})=\alpha\sigma^{2}+\alpha\theta\sigma^{2}=\alpha (1+\theta)\sigma ^{2}.$

Finally, using (10)

$latex \gamma_{2}=\alpha^{2}\gamma_{2}+\alpha(\alpha+\theta)\sigma^{2}+\alpha^2\theta(1 +\theta)\sigma^{2}=\alpha^{2}\gamma_{2}+\alpha\sigma^2 (\alpha+\theta +\alpha\theta+\alpha\theta^2)\sigma^{2},$

$latex \gamma_{2}=\frac{\alpha\sigma^2 (\alpha+\theta +\alpha\theta +\alpha\theta^2)\sigma^{2}}{1-\alpha^{2}},$

$latex \rho _{2}=\frac{\alpha\sigma^2 (\alpha+\theta+\alpha\theta+\alpha\theta^2)}{1+2\alpha \theta+\theta^{2}}.$

A couple of errors have been corrected on June 22, 2021. Hope this is final.