Thomas M.Cover Joy A.Thomas著, 美国信息论经典教材

信息论的经典教材，斯坦福教授Cover的代表作，在经济，通信，计算机领域均有帮助

机器学习的信息理论基础。 我移除了里面添加的链接等信息，可以直接打印成很干净的书。

This tutorial is concerned with applications of information theory concepts in statistics, in the finite alphabet setting. The information measure known as information divergence or Kullback-Leibler distance or relative entropy plays a key role, often with a geometric flavor as an analogue of squared Euclidean distance, as in the concepts of I-projection, I-radius and I-centroid. The topics covered include large deviations, hypothesis testing, maximum likelihood estimation in exponential families, analysis of contingency tables, and iterative algorithms with an “information geometry” background. Also, an introduction is provided to the theory of universal coding, and to statistical inference via the minimum description length principle motivated by that theory.

出版社：Springer 作者：Raymond W. Yeung 书名：Information Theory and Network Coding

Information theory answers two fundamental questions in communication theory: What is the ultimate data compression (answer: the entropy H), and what is the ultimate transmission rate of communication (answer: the channel capacity C). For this reason some consider information theory to be a subset of communication theory. We argue that it is much more. Indeed, it has fundamental contributions to make in statistical physics (thermodynamics), computer science (Kolmogorov complexity or algorithmic complexity), statistical inference (Occam’s Razor: “The simplest explanation is best”), and to probability and statistics (error exponents for optimal hypothesis testing and estimation)

elements of information theory

信息论英文原版书籍第二版，elements of information theory

此工具箱包含用于计算以下数量的离散随机变量的函数： 1)熵2)联合熵3）条件熵4）相对熵（KL散度） 5)相互信息6）归一化互信息7)归一化变异信息这个包现在是 PRML 工具箱的一部分 ( http://www.mathworks.com/matlabcentral/fileexchange/55826-pattern-recognition-and-machine-learning-toolbox )。

What is information? How do we quantify or measure the amount of information that is present in a file of data, or a string of text? How do we encode the information so that it can be stored efficiently, or transmitted reliably?