Zero-inflated model

In statistics, a zero-inflated model is a statistical model based on a zero-inflated probability distribution, i.e. a distribution that allows for frequent zero-valued observations.

Zero-inflated Poisson[]

One well-known zero-inflated model is Diane Lambert's zero-inflated Poisson model, which concerns a random event containing excess zero-count data in unit time.^[1] For example, the number of insurance claims within a population for a certain type of risk would be zero-inflated by those people who have not taken out insurance against the risk and thus are unable to claim. The zero-inflated Poisson (ZIP) model mixes two zero generating processes. The first process generates zeros. The second process is governed by a Poisson distribution that generates counts, some of which may be zero. The mixture is described as follows:

${\displaystyle \Pr(Y=0)=\pi +(1-\pi )e^{-\lambda }}$

${\displaystyle \Pr(Y=y_{i})=(1-\pi ){\frac {\lambda ^{y_{i}}e^{-\lambda }}{y_{i}!}},\qquad y_{i}=1,2,3,...}$

where the outcome variable ${\displaystyle y_{j}}$ has any non-negative integer value, ${\displaystyle \lambda }$ is the expected Poisson count for the ${\displaystyle i}$th individual; ${\displaystyle \pi }$ is the probability of extra zeros.

The mean is ${\displaystyle (1-\pi )\lambda }$ and the variance is ${\displaystyle \lambda (1-\pi )(1+\pi \lambda )}$.

Estimators of ZIP parameters[]

The method of moments estimators are given by^[2]

${\displaystyle {\hat {\lambda }}_{mo}={\frac {s^{2}+m^{2}}{m}}-1,}$

${\displaystyle {\hat {\pi }}_{mo}={\frac {s^{2}-m}{s^{2}+m^{2}-m}},}$

where ${\displaystyle m}$ is the sample mean and ${\displaystyle s^{2}}$ is the sample variance.

The maximum likelihood estimator^[3] can be found by solving the following equation

${\displaystyle m(1-e^{-{\hat {\lambda }}_{ml}})={\hat {\lambda }}_{ml}\left(1-{\frac {n_{0}}{n}}\right).}$

where ${\displaystyle {\frac {n_{0}}{n}}}$ is the observed proportion of zeros.

A closed form solution of this equation is given by^[4]

${\displaystyle {\hat {\lambda }}_{ml}=W_{0}(-se^{-s})+s}$

with ${\displaystyle W_{0}}$ being the main branch of Lambert's W-function^[5] and

${\displaystyle s={\frac {m}{1-{\frac {n_{0}}{n}}}}}$.

Alternatively, the equation can be solved by iteration.^[6]

The maximum likelihood estimator for ${\displaystyle \pi }$ is given by

${\displaystyle {\hat {\pi }}_{ml}=1-{\frac {m}{{\hat {\lambda }}_{ml}}}.}$

Related models[]

In 1994, Greene considered the zero-inflated negative binomial (ZINB) model.^[7] Daniel B. Hall adapted Lambert's methodology to an upper-bounded count situation, thereby obtaining a zero-inflated binomial (ZIB) model.^[8]

Discrete pseudo compound Poisson model[]

If the count data ${\displaystyle Y}$ is such that the probability of zero is larger than the probability of nonzero, namely

${\displaystyle \Pr(Y=0)>0.5}$

then the discrete data ${\displaystyle Y}$ obey discrete pseudo compound Poisson distribution.^[9]

In fact, let ${\displaystyle G(z)=\sum \limits _{n=0}^{\infty }P(Y=n)z^{n}}$ be the probability generating function of ${\displaystyle y_{i}}$. If ${\displaystyle p_{0}=\Pr(Y=0)>0.5}$, then ${\displaystyle |G(z)|\geqslant p_{0}-\sum \limits _{i=1}^{\infty }p_{i}=2p_{0}-1>0}$. Then from the Wiener–Lévy theorem,^[10] ${\displaystyle G(z)}$ has the probability generating function of the discrete pseudo compound Poisson distribution.

We say that the discrete random variable ${\displaystyle Y}$ satisfying probability generating function characterization

${\displaystyle G_{Y}(z)=\sum \limits _{n=0}^{\infty }P(Y=n)z^{n}=\exp \left(\sum _{k=1}^{\infty }\alpha _{k}\lambda (z^{k}-1)\right),\quad (|z|\leq 1)}$

has a discrete pseudo compound Poisson distribution with parameters

${\displaystyle (\lambda _{1},\lambda _{2},\ldots )=(\alpha _{1}\lambda ,\alpha _{2}\lambda ,\ldots )\in \mathbb {R} ^{\infty }\left(\sum _{k=1}^{\infty }\alpha _{k}=1,\sum \limits _{k=1}^{\infty }|\alpha _{k}|<\infty ,\alpha _{k}\in \mathbb {R} ,\lambda >0\right).}$

When all the ${\displaystyle \alpha _{k}}$ are non-negative, it is the discrete compound Poisson distribution (non-Poisson case) with overdispersion property.

See also[]

• Poisson distribution
• Zero-truncated Poisson distribution
• Compound Poisson distribution
• Sparse approximation
• Hurdle model

Software[]

• pscl and brms R packages

References[]