## You are here

Homerecurrence in a Markov chain

## Primary tabs

# recurrence in a Markov chain

Let $\{X_{n}\}$ be a stationary Markov chain and $I$ the state space. Given $i,j\in I$ and any non-negative integer $n$, define a number $F_{{ij}}^{n}$ as follows:

$F_{{ij}}^{n}:=\begin{cases}0&\text{if }n=0,\\ P(X_{n}=j\mbox{ and }X_{m}\neq j\mbox{ for }0<m<n\mid X_{0}=i)&\text{otherwise% }.\end{cases}$ |

In other words, $F_{{ij}}^{n}$ is the probability that the process *first* reaches state $j$ at time $n$ from state $i$ at time $0$.

From the definition of $F_{{ij}}^{n}$, we see that the probability of the process reaching state $j$ *within and including* time $n$ from state $i$ at time $0$ is given by

$\sum_{{m=0}}^{n}F_{{ij}}^{m}.$ |

As $n\to\infty$, we have the limiting probability of the process reaching $j$ *eventually* from the initial state of $i$ at $0$, which we denote by $F_{{ij}}$:

$F_{{ij}}:=\sum_{{m=0}}^{{\infty}}F_{{ij}}^{m}.$ |

Definitions. A state $i\in I$ is said to be *recurrent* or *persistent* if $F_{{ii}}=1$, and *transient* otherwise.

Given a recurrent state $i$, we can further classify it according to “how soon” the state $i$ returns after its initial appearance. Given $F_{{ii}}^{n}$, we can calculate the expected number of steps or transitions required to *return* to state $i$ by time $n$. This expectation is given by

$\sum_{{m=0}}^{n}mF_{{ii}}^{m}.$ |

When $n\to\infty$, the above expression may or may not approach a limit. It is the expected number of transitions needed to return to state $i$ *at all* from the beginning. We denote this figure by $\mu_{i}$:

$\mu_{{i}}:=\sum_{{m=0}}^{{\infty}}mF_{{ii}}^{m}.$ |

Definitions. A recurrent state $i\in I$ is said to be *positive* or *strongly ergodic* if $\mu_{i}<\infty$, otherwise it is called *null* or *weakly ergodic*. If a stronly ergodic state is in addition aperiodic, then it is said to be an *ergodic state*.

## Mathematics Subject Classification

60J10*no label found*

- Forums
- Planetary Bugs
- HS/Secondary
- University/Tertiary
- Graduate/Advanced
- Industry/Practice
- Research Topics
- LaTeX help
- Math Comptetitions
- Math History
- Math Humor
- PlanetMath Comments
- PlanetMath System Updates and News
- PlanetMath help
- PlanetMath.ORG
- Strategic Communications Development
- The Math Pub
- Testing messages (ignore)

- Other useful stuff
- Corrections