\magnification 2200
\parindent 0pt
\parskip 13pt
\nopagenumbers
\hsize 7.5truein
\vsize 9.9 truein
\hoffset -0.45truein
\def\u{\vskip -10pt}
\def\w{\vskip -8pt}
\input ../../../PAPERS/GOLD/psfig


The $ij^{th}$ entry of $P^k$ is the probability that you are in the jth state after exactly $k$ steps given that you started in the ith state.

Suppose that $p_i$ is the probability that you start in state $i$.  Let $p = (p_1, ..., p_n)$.  Then $pP^k = (s_1, ..., s_n)$ where $s_j$ is the probability that you are in the jth state after exactly $k$ steps.

{\bf If there are transient states:}

If $P = \left(\matrix{I & 0 \cr R & Q}\right)$, then can use $Q^k$ instead of $P^k$.


If $Q$ represents transient states, then $lim_{n \rightarrow \infty} Q^n = 0$

If $N = (I - Q)^{-1} = \Sigma_{n=0}^\infty Q^n$, then the $ij^{th}$ entry of $N$ is the expected 
number of times you are in state $j$ given that you started in state $i$.

Hence the expected number of steps before absorption is the sum of the ith row of $(I - Q)^{-1} $ given that you started in state $i$.

If $B = NR = (I - Q)^{-1}R$, then $b_{ij}$ is the probability that absorbed in state $j$ given that 
you started in state $i$.

{\bf Ergodic (there are NO transient states):}
\w
{\it If regular (i.e. $P^k$ is a positive matrix for some $k$):}
\w
$P^n$ is a positive matrix for all large $n$ ($n \geq k$).
\w
$lim_{n \rightarrow \infty}P^n = W = \left(\matrix{{\bf w} \cr . \cr . \cr . \cr {\bf w} }\right)$
\w\w
$lim_{n \rightarrow \infty}pP^n = {\bf w}$
\w\vskip -3pt
${\bf w}P = {\bf w}$
\w
If $E = (I - Z + JZ_{dg})D$, then $e_{ij}$ is the expected number of steps from state $i$ to state $j$ (without going through state $j$ in between, i.e., first time getting to/returning to state $j$).
\hskip 1in $ e_{ii} = {1 \over w_i}$
\w
Regular if and only if period = 1.

{\it If not regular}
\w
Period $ > 1$
\w
Fix $i$:  $d$ = Period = gcd$\{ n ~|~ $ there is a path from $u_i$ to $u_i$ of length $n \}$
\w
States can be partitioned into $d$ periodic classes, $C_0, ..., C_{d-1}$ such that if you start at a vertex in $C_i$, then after $k$ steps, you are in a vertex in $C_{i + k (mod d)}.






\end