A readily accessible introduction for students of communication theory, computer science and statistics. Complete with numerous illustrations, original problems and historical notes, it will also prove an invaluable practical reference for communications professionals and statisticians specializing in information theory.

Contents:

• Entropy
• Relative Entropy and Mutual Information
• The Asymptotic Equipartition Property
• Entropy Rates of a Stochastic Process
• Data Compression
• Gambling and Data Compression
• Kolmogorov Complexity
• Channel Capacity
• Differential Entropy
• The Gaussian Channel
• Maximum Entropy and Spectral Estimation
• Information Theory and Statistics
• Rate Distortion Theory
• Network Information Theory
• Information Theory and the Stock Market
• Inequalities in Information Theory
• Bibliography
• Index