maximum likelihood

The maximum likelihood estimator of a parameter $\theta$ is the value that maximizes $L(\theta |x)$:

$$L(\hat{\theta }|x)\ge L(\theta |x),\forall \theta \in \Omega$$

i.e., MLE finds the parameter values that make the observed data most probable.

Note: MLEs…

• may not be unique
• may not be differentiable
• may not exist

Steps:

• find the log-likelihood function to make differentiation easier
• find parameter value(s) that maximize the log-likelihood function: $\hat{\theta }=\arg {\max }_{\theta }\ell (\theta \mid x)$
  • compute the score (1st order derivative), set to 0, solve
  • check that the 2nd order derivative < 0

Invariance property: if $\hat{\theta }$ is the MLE and $\psi (\theta )$ is 1-1 on $\Omega$, then $\psi (\hat{\theta })$ is the MLE of $\psi (\theta )$.