External Resources

External Resources

A curated list of external references, papers, software libraries, and other useful links related to entropy, information theory, and statistical measurement.

Key Papers & Foundational Works

• Shannon, C. E. (1948). A Mathematical Theory of Communication
• Cover, T. M., & Thomas, J. A. (2012). Elements of Information Theory
• Jaynes, E. T. (1957). Information Theory and Statistical Mechanics
• Paninski, L. (2003). Estimation of entropy and mutual information. Neural Computation, 15(6), 1191–1253.

Software Libraries & Toolkits

• Scipy.stats.entropy(https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html) – Built-in entropy and KL divergence in SciPy.
• Antropy(https://raphaelvallat.com/antropy/) – Python library for time series entropy: sample, permutation, spectral entropy, etc.
• EntropyHub(http://www.entropyhub.xyz/) – Multi-language (Python/Matlab/Julia) toolkit for advanced entropy measures.
• PyEntrp – Python library for entropy in time series analysis.

Online Courses & Tutorials

• Information Theory (Video Lectures) – MIT OpenCourseWare](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-441-information-theory-spring-2016/video-lectures/)

Recommended Books

• Cover, T. M. & Thomas, J. A., Elements of Information Theory
• MacKay, D. J. C., Information Theory, Inference and Learning Algorithms
• Roman, S., Coding and Information Theory

Blog Articles & Further Reading

• Quanta Magazine: How Shannon Entropy Imposes Fundamental Limits on Communication
• Entropy in Decision Trees (Towards Data Science)](https://towardsdatascience.com/entropy-how-decision-trees-make-decisions-2946b9c18c8)
• Wikipedia

Community & Forums

• Cross Validated – Information Theory Questions(https://stats.stackexchange.com/questions/tagged/information-theory)
• Stack Overflow – Entropy Tag(https://stackoverflow.com/questions/tagged/entropy)