Video of the day!
This is a very cool and intuitive video showing behaviors of MH and HMC samplers!
I wish I had more time and write write up about MCMC and sampling methods but for now, I'd enjoy this video. (best enjoyed w/ sound)
Anyway, the beauty is they only need to evaluate the relative probabilities of two states and there's no hard limit on state spaces. (Yet, you wouldn't know how long you have to wait and there's an asymptotic guarantee that the converged heuristics are reasonable.)
P.S. Human inference as a kind of MCMC process
Marr's three levels (how the mind and brain implement Bayesian inference (level 2, 3))
MCMC algorithms for mapping out mental representations: (paper)
Rule based concept learning: (paper)
P.S.2.
Human cognition has to be fast.
# Trade-off between inference & computation
Even just one or a few posterior samples are very useful in the settings that matter most for everyday cognition. (which is very different from a statistician's perspective on sampling/inference.)
One and Done? (paper)