Scoring algorithm
Scoring algorithm, also known as Fisher's scoring,[1] is a form of Newton's method used in statistics to solve maximum likelihood equations numerically, named after Ronald Fisher.
Sketch of derivation[]
Let be random variables, independent and identically distributed with twice differentiable p.d.f. , and we wish to calculate the maximum likelihood estimator (M.L.E.) of . First, suppose we have a starting point for our algorithm , and consider a Taylor expansion of the score function, , about :
where
is the observed information matrix at . Now, setting , using that and rearranging gives us:
We therefore use the algorithm
and under certain regularity conditions, it can be shown that .
Fisher scoring[]
In practice, is usually replaced by , the Fisher information, thus giving us the Fisher Scoring Algorithm:
- ..
See also[]
References[]
- ^ Longford, Nicholas T. (1987). "A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects". Biometrika. 74 (4): 817–827. doi:10.1093/biomet/74.4.817.
Further reading[]
- Jennrich, R. I. & Sampson, P. F. (1976). "Newton-Raphson and Related Algorithms for Maximum Likelihood Variance Component Estimation". Technometrics. 18 (1): 11–17. doi:10.1080/00401706.1976.10489395.
- Maximum likelihood estimation