7.3 The Metropolis-Hastings algorithm

  • step 1: ok : \(q(\mu_{proposal}|\mu)\)

  • step 2:

    • we are going to calculate an acceptance probability:

    \[\alpha = \{1, \frac{f(\mu_{proposal}) L(\mu_{proposal}|y) q(\mu|\mu_{proposal})}{f(\mu)L(\mu|y) q(\mu_{proposal|\mu})} \}\]

Because the Uniform proposal model is symmetric:

\[ q(\mu_{proposal}|\mu) = q(\mu|\mu_{proposal})\]

Then after multiplying by \(f(y)\) we have now:

\[ \alpha = min\{1, \frac{f(\mu_{proposal}|y)}{f(\mu|y)} \} \]

This ration is equivalent to the unnormalized posterior.

Two scnearios:

  • Scenario 1: \(f(\mu_{proposal}|y) \geq f(\mu|y)\) -> \(\alpha = 1\) we are moving

  • Scenario 2: $f(_{proposal}|y) < f(|y) $ then we move according to the probability \(\alpha\)

one_mh_iteration <- function(w, current){
 # STEP 1: Propose the next chain location
 proposal <- runif(1, min = current - w, max = current + w)
  
 # STEP 2: Decide whether or not to go there
 proposal_plaus <- dnorm(proposal, 0, 1) * dnorm(6.25, proposal, 0.75)
 current_plaus  <- dnorm(current, 0, 1) * dnorm(6.25, current, 0.75)
 alpha <- min(1, proposal_plaus / current_plaus)
 next_stop <- sample(c(proposal, current), 
                     size = 1, prob = c(alpha, 1-alpha))
  
 # Return the results
 return(data.frame(proposal, alpha, next_stop))
}

set.seed(8)
one_mh_iteration(w = 1, current = 3)
##   proposal     alpha next_stop
## 1  2.93259 0.8240205   2.93259