Markov Decision Processes Discrete Stochastic Dynamic Programming

详细的介绍MCMC方法及其应用，对于想深入学习MCMC的你有很大的帮助，推荐一下.附阅读器

该文档主要讲解马尔可夫聚类算法（The Markov Cluster Algorithm，MCL），配有计算公式，转化方法，结合实例讲解算法过程，个人感觉思路很清晰，讲解的很详细。

《图像分析中的马尔可夫随机场模型》 (Springer 1995, 2nd edition 2001, 3rd edition 2009) 被誉为"图像分析领域里程碑意义的工作"。

此程序主要来源于《基于Markov随机场的小波域图像建模及分割:Matlab环境》 （刘国英等编著），并配有原书详尽代码解释，此外还有自己试验样图。很适合入门奥

一个简单的用MATLAB仿真markov链的思想

Hidden Markov Models (HMMs) provide a simple and effective framework for modelling time-varying spectral vector sequences. As a consequence, almost all present day large vocabulary continuous speech recognition (LVCSR) systems are based on HMMs. Whereas the basic principles underlying HMM-based LVCSR are rather straightforward, the approximations and simplifying assumptions involved in a direct implementation of these principles would result in a system which has poor accuracy and unacceptable sensitivity to changes in operating environment. Thus, the practical application of HMMs in modern systems involves considerable sophistication. The aim of this review is first to present the core architecture of a HMM-based LVCSR system and then describe the various refinements which are needed to achieve state-of-the-art performance. These refinements include feature projection, improved covariance modelling, discriminative parameter estimation, adaptation and normalisation, noise compensation and multi-pass system combination. The review concludes with a case study of LVCSR for Broadcast News and Conversation transcription in order to illustrate the techniques described.