Hidden Markov model

Difference between a mixture model and HHM

  • If we examine a single time slice of the model, it can be seen as a mixture distribution with component densities given by p(X|Z)
  • It can be interpreted as an extension of a mixture model where the choice of mixture component for each observation is not independent but depends on the choice of component for the previous observations  (p(Z_n|Z_{n-1}))


  • Speech recognition
  • Natural language modeling
  • On-line handwriting recognition
  • analysis of biological sequences such as protein and DNA

Transition probability

  • Latent variables; discrete multinomial variables Z_n = describe which component of the mixture is responsible for generating the corresponding observation X_n
  • The probability distribution of Z_n depends on the previous latent variable Z_{n-1} through conditional distribution p(Z_n|Z_{n-1})
  • Conditional distribution

p(Z_n|Z_{n-1}, A) = \displaystyle \prod_{k=1}^{K}\prod_{j=1}^{K} A_{jk}^{Z_{n-1, j}Z_{nk}}

  • Inital latent node z_1 does not have a parent node, so it has a marginal distribution

p(Z_1|\pi) = \displaystyle \prod_{k=1}^{K} \pi_{k}^{z_{1k}}

  • Lattice or trellis diagram

Emission probability


  • Three Gaussian distribution/ two dice problem
  • Handwriting

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s