Maximum likelihood: idea of the method and application to life of a bulb. Sometimes I plagiarize from my book.

### Maximum likelihood idea

The main idea of the maximum likelihood (ML) method is illustrated in Figure 1. We start with the sample depicted with points on the horizontal axis. Then we think which of the densities shown on the figure is more likely to have generated that sample. Of course, it's the one on the left, filled with grey.

This density takes higher values at observed points than the other two. Note also that the position of the density is regulated by its parameters. This explains the **main idea**: choose the parameters so as to maximize the density at the observed points.

### Algorithm

**Step 1**. A statistical model usually contains a random term. To describe that term, choose a density from some parametric family. Denote it

**Step 2**. Assume that observations are independent. Then the joint density is a product of own densities:

**Definition**. The joint density as a function of just parameters is called a **likelihood function** and denoted **maximum likelihood estimators**, and thus the name *Maximum Likelihood* (ML) method.

**Step 3**. Since **log-likelihood function**

**Comments**. (1) Most of the time the likelihood function is difficult to maximize analytically. Then maximization is done on the computer. A numerical algorithm can give the solution only approximately. Moreover, the likelihood function may not have maximums at all or may have many maximums; in the former case the numerical procedure does not converge and in the latter the computer gives only one solution.

(2) One should distinguish *models* and *estimation methods*. The OLS method applied to the linear model gives OLS estimators. The ML method applied to the same linear model gives ML estimators. Most linear models are dealt with using the least squares method. All exercises for maximum likelihood require some algebra, as is seen from the algorithm.

### Example: life of a bulb

Life of a bulb is described by the **exponential distribution**

where

**Exercise**. Derive the ML estimator of

**Solution**. Step 1.

Step 2. Assuming independent observations, the joint density is a product of these densities

Step 3. The log-likelihood function is

**Conclusion**:

## Leave a Reply

You must be logged in to post a comment.