Raising the bar

What is a stationary process? More precisely, this post is about a discrete weakly stationary process. This topic is not exactly a beginners Stats, I am posting this to help those who study Econometrics using Introduction to Econometrics, by Christopher Dougherty, published by Oxford University Press, UK, in 2016.

Point of view. At discrete moments in time , we observe some random variables .  can be, for example, periodical temperature measurements in a certain location. You can imagine a straight line, with moments labeled on it, and for each , some variable  attached to it. In general,  may have different distributions and in theory time moments may extend infinitely to the left and right.

Definition. We say that the collection is (weakly) stationary if it satisfies three conditions:

1. The means are constant (that is, do not depend on ),
2. The variances are also constant (same thing, they do not depend on ), and
3. The covariances depend only on the distance in time between two moments .

Regarding the last condition, recall the visualization of the process, with random variables sticking out of points in time, and the fact that the distance between two moments  is given by the absolute value . The condition  says that the covariance between  is some (unspecified) function of this distance. It should not depend on any of the moments  themselves.

If you want a complex definition to stay in your memory, you have to chew and digest it. The best thing to do is to prove a couple of properties.

Main property. A sum of two independent stationary processes is also stationary.

Proof. The assumption is that each variable in the collection  is independent of each variable in the collection . We need to check that  satisfies the definition of a stationary process.

Obviously,  is constant.

Similarly, by independence we have , so variance of the sum is constant.

Finally, using properties of covariance,

(two terms disappear by independence)

(each covariance depends only on , so their sum depends only on ).

Conclusion. You certainly know that . The above property is similar to this:

stationary process + stationary process = stationary process

(under independence). Now you can understand the role of stationary processes in the set of all processes: they play the role of zero. That is to say, the process is not very different from the process if their difference is stationary.

Generalization. Any linear combination of independent stationary processes is stationary.