(generated image for illustration only)
The Metropolis-Hastings algorithm is a prominent MCMC method for obtaining a sequence of random samples from a probability distribution for which direct sampling is difficult. At each iteration, it generates a candidate for the next sample based on the current sample. This candidate is then accepted or rejected with a certain probability, ensuring the resulting chain converges to the desired distribution.
The algorithm works by constructing a Markov chain whose stationary distribution is the target distribution \(P(x)\). Let the current state be \(x_t\). First, a candidate state \(x’\) is generated from a proposal distribution \(Q(x’|x_t)\), which can be any distribution that is easy to sample from (e.g., a Gaussian centered at \(x_t\)). Second, an acceptance probability \(\alpha(x’, x_t)\) is calculated: \(\alpha(x’, x_t) = \min\left(1, \frac{P(x’)Q(x_t|x’)}{P(x_t)Q(x’|x_t)}\right)\). A random number \(u\) is drawn from a uniform distribution on \([0, 1]\). If \(u \le \alpha\), the candidate is accepted, and the next state is set to \(x_{t+1} = x’\). Otherwise, the candidate is rejected, and the chain remains at the current state, \(x_{t+1} = x_t\).
The brilliance of this acceptance ratio is that it only requires knowing \(P(x)\) up to a constant of proportionality, since any normalization constant cancels out in the fraction \(\frac{P(x’)}{P(x_t)}\). This is crucial in Bayesian statistics, where the posterior distribution is often known only up to the intractable marginal likelihood. The original Metropolis algorithm (1953) is a special case where the proposal distribution \(Q\) is symmetric, i.e., \(Q(x’|x_t) = Q(x_t|x’)\), simplifying the acceptance probability to \(\min\left(1, \frac{P(x’)}{P(x_t)}\right)\). Hastings’ 1970 generalization allowed for asymmetric proposal distributions, significantly broadening the algorithm’s applicability and efficiency.
Due to scrapping bot traffic, currently more than 40k per day, this content is reserved to community members.
> Login < or > Register < (100% free) to access this, so as all other restricted content and tools.
Metropolis-Hastings Algorithm
(if date is unknown or not relevant, e.g. "fluid mechanics", a rounded estimation of its notable emergence is provided)
Full size images and downloads are only available, 100% free, for registered members.
> Login <
