recurrence in a Markov chain

Let {Xn} be a stationary ( Markov chainMathworldPlanetmath and I the state spacePlanetmathPlanetmath. Given i,jI and any non-negative integer n, define a number Fijn as follows:

Fijn:={0if n=0,P(Xn=j and Xmj for 0<m<nX0=i)otherwise.

In other words, Fijn is the probability that the process first reaches state j at time n from state i at time 0.

From the definition of Fijn, we see that the probability of the process reaching state j within and including time n from state i at time 0 is given by


As n, we have the limiting probability of the process reaching j eventually from the initial state of i at 0, which we denote by Fij:


Definitions. A state iI is said to be recurrent or persistent if Fii=1, and transient otherwise.

Given a recurrent state i, we can further classify it according to “how soon” the state i returns after its initial appearance. Given Fiin, we can calculate the expected number of steps or transitions required to return to state i by time n. This expectation is given by


When n, the above expression may or may not approach a limit. It is the expected number of transitions needed to return to state i at all from the beginning. We denote this figure by μi:


Definitions. A recurrent state iI is said to be or strongly ergodic if μi<, otherwise it is called null or weakly ergodic. If a stronly ergodic state is in additionPlanetmathPlanetmath aperiodic (, then it is said to be an ergodic state.

Title recurrence in a Markov chain
Canonical name RecurrenceInAMarkovChain
Date of creation 2013-03-22 16:24:43
Last modified on 2013-03-22 16:24:43
Owner CWoo (3771)
Last modified by CWoo (3771)
Numerical id 5
Author CWoo (3771)
Entry type Definition
Classification msc 60J10
Synonym null recurrent
Synonym positive recurrent
Synonym strongly ergodic
Synonym weakly ergodic
Defines recurrent state
Defines persistent state
Defines transient state
Defines null state
Defines positive state
Defines ergodic state