# Hidden Markov model

Difference between a mixture model and HHM

• If we examine a single time slice of the model, it can be seen as a mixture distribution with component densities given by $p(X|Z)$
• It can be interpreted as an extension of a mixture model where the choice of mixture component for each observation is not independent but depends on the choice of component for the previous observations  ($p(Z_n|Z_{n-1})$)

Applications

• Speech recognition
• Natural language modeling
• On-line handwriting recognition
• analysis of biological sequences such as protein and DNA

Transition probability

• Latent variables; discrete multinomial variables $Z_n$ = describe which component of the mixture is responsible for generating the corresponding observation $X_n$
• The probability distribution of $Z_n$ depends on the previous latent variable $Z_{n-1}$ through conditional distribution $p(Z_n|Z_{n-1})$
• Conditional distribution

$p(Z_n|Z_{n-1}, A) = \displaystyle \prod_{k=1}^{K}\prod_{j=1}^{K} A_{jk}^{Z_{n-1, j}Z_{nk}}$

• Inital latent node $z_1$ does not have a parent node, so it has a marginal distribution

$p(Z_1|\pi) = \displaystyle \prod_{k=1}^{K} \pi_{k}^{z_{1k}}$

• Lattice or trellis diagram

Emission probability

Example;

• Three Gaussian distribution/ two dice problem
• Handwriting