6. Markov Models
Learning objectives
- Discuss the Markov Property
- Introduce MCMC
Tabular State Space
fairy tale generator
image credit: Aja Hammerly
Trajectories
once, upon, a, time, a, bird, and, a, mouse
a, sausage, entered, into, a, partnership, and, set
bird, a, and, set, up, house, together
Markov Property
The future of a stochastic process is independent of its past
P(Xt+1=x|Xt,Xt−1,...,Xt−k)=P(Xt+1=x|Xt) * memoryless property
Metropolis-Hastings
Metropolis-Hastings Algorithm
Markov Chain Monte Carlo
MCMC to posterior dist
6. Markov Models