Markov chains on a measurable state space

A Markov chain on a measurable state space is a discrete-time-homogeneous Markov chain with a measurable space as state space.

History[]

The definition of Markov chains has evolved during the 20th century. In 1953 the term Markov chain was used for stochastic processes with discrete or continuous index set, living on a countable or finite state space, see Doob.^[1] or Chung.^[2] Since the late 20th century it became more popular to consider a Markov chain as a stochastic process with discrete index set, living on a measurable state space.^[3][4][5]

Definition[]

Denote with ${\displaystyle (E,\Sigma )}$ a measurable space and with ${\displaystyle p}$ a Markov kernel with source and target ${\displaystyle (E,\Sigma )}$. A stochastic process ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ on ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} )}$ is called a time homogeneous Markov chain with Markov kernel ${\displaystyle p}$ and start distribution ${\displaystyle \mu }$ if

${\displaystyle \mathbb {P} [X_{0}\in A_{0},X_{1}\in A_{1},\dots ,X_{n}\in A_{n}]=\int _{A_{0}}\dots \int _{A_{n-1}}p(y_{n-1},A_{n})\,p(y_{n-2},dy_{n-1})\dots p(y_{0},dy_{1})\,\mu (dy_{0})}$

is satisfied for any ${\displaystyle n\in \mathbb {N} ,\,A_{0},\dots ,A_{n}\in \Sigma }$. One can construct for any Markov kernel and any probability measure an associated Markov chain.^[4]

Remark about Markov kernel integration[]

For any measure ${\displaystyle \mu \colon \Sigma \to [0,\infty ]}$ we denote for ${\displaystyle \mu }$-integrable function ${\displaystyle f\colon E\to \mathbb {R} \cup \{\infty ,-\infty \}}$ the Lebesgue integral as ${\displaystyle \int _{E}f(x)\,\mu (dx)}$. For the measure ${\displaystyle \nu _{x}\colon \Sigma \to [0,\infty ]}$ defined by ${\displaystyle \nu _{x}(A):=p(x,A)}$ we used the following notation:

${\displaystyle \int _{E}f(y)\,p(x,dy):=\int _{E}f(y)\,\nu _{x}(dy).}$

Basic properties[]

Starting in a single point[]

If ${\displaystyle \mu }$ is a Dirac measure in ${\displaystyle x}$, we denote for a Markov kernel ${\displaystyle p}$ with starting distribution ${\displaystyle \mu }$ the associated Markov chain as ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ on ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} _{x})}$ and the expectation value

${\displaystyle \mathbb {E} _{x}[X]=\int _{\Omega }X(\omega )\,\mathbb {P} _{x}(d\omega )}$

for a ${\displaystyle \mathbb {P} _{x}}$-integrable function ${\displaystyle X}$. By definition, we have then ${\displaystyle \mathbb {P} _{x}[X_{0}=x]=1}$.

We have for any measurable function ${\displaystyle f\colon E\to [0,\infty ]}$ the following relation:^[4]

${\displaystyle \int _{E}f(y)\,p(x,dy)=\mathbb {E} _{x}[f(X_{1})].}$

Family of Markov kernels[]

For a Markov kernel ${\displaystyle p}$ with starting distribution ${\displaystyle \mu }$ one can introduce a family of Markov kernels ${\displaystyle (p_{n})_{n\in \mathbb {N} }}$ by

${\displaystyle p_{n+1}(x,A):=\int _{E}p_{n}(y,A)\,p(x,dy)}$

for ${\displaystyle n\in \mathbb {N} ,\,n\geq 1}$ and ${\displaystyle p_{1}:=p}$. For the associated Markov chain ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ according to ${\displaystyle p}$ and ${\displaystyle \mu }$ one obtains

${\displaystyle \mathbb {P} [X_{0}\in A,\,X_{n}\in B]=\int _{A}p_{n}(x,B)\,\mu (dx)}$.

Stationary measure[]

A probability measure ${\displaystyle \mu }$ is called stationary measure of a Markov kernel ${\displaystyle p}$ if

${\displaystyle \int _{A}\mu (dx)=\int _{E}p(x,A)\,\mu (dx)}$

holds for any ${\displaystyle A\in \Sigma }$. If ${\displaystyle (X_{n})_{n\in \mathbb {N} }}$ on ${\displaystyle (\Omega ,{\mathcal {F}},\mathbb {P} )}$ denotes the Markov chain according to a Markov kernel ${\displaystyle p}$ with stationary measure ${\displaystyle \mu }$, and the distribution of ${\displaystyle X_{0}}$ is ${\displaystyle \mu }$, then all ${\displaystyle X_{n}}$ have the same probability distribution, namely:

${\displaystyle \mathbb {P} [X_{n}\in A]=\mu (A)}$

for any ${\displaystyle A\in \Sigma }$.

Reversibility[]

A Markov kernel ${\displaystyle p}$ is called reversible according to a probability measure ${\displaystyle \mu }$ if

${\displaystyle \int _{A}p(x,B)\,\mu (dx)=\int _{B}p(x,A)\,\mu (dx)}$

holds for any ${\displaystyle A,B\in \Sigma }$. Replacing ${\displaystyle A=E}$ shows that if ${\displaystyle p}$ is reversible according to ${\displaystyle \mu }$, then ${\displaystyle \mu }$ must be a stationary measure of ${\displaystyle p}$.

See also[]

• Harris chain
• Subshift of finite type

References[]