ITNuggets
material surrounding the talk "Nuggets of Shannon Information Theory" given by Christian Schaffner at MCH2022. Recording available at https://www.youtube.com/watch?v=we7T7NqTrlU
Folders
- presentation contains the Powerpoint presentation, and the downloaded video of juggling Shannon
- notebooks contains the main ITNuggets.ipynb Jupyter notebook and related material
- articles contains Shannon's original article
Consulted material & Attributions
Scientific material
- A mathematical theory of communication - the original paper by Claude Shannon, published in 1948 in The Bell System Technical Journal. Check its wikipedia entry
- Course lecture notes (work-in-progress) by Yfke Dulek and Christian Schaffner, available on https://github.com/cschaffner/InformationTheory/raw/master/Script/InfTheory3.pdf
- Thomas M. Cover, Joy A. Thomas. 'Elements of information theory': http://onlinelibrary.wiley.com/book/10.1002/0471200611, 2nd Edition. New York: Wiley-Interscience, 2006. ISBN 0-471-24195-4.
- David J. C. MacKay. 'Information Theory, Inference, and Learning Algorithms': http://www.inference.phy.cam.ac.uk/mackay/itila/book.html. Cambridge University Press, 2003. ISBN 0-521-64298-1
- Michael Walter's course homepage of his course "Introduction to Information Theory (Fall 2021)".
- Entropy Diagram for Two Random Variables
- Elements of Information Theory, ECE 534 by Natasha Devroye, University of Illinois at Chicago
Icons
Claude Shannon
Videos about Logarithms
- How to program your own Log() function, python code
- How they Invented Logarithms
- Why The Logarithm Is So Important For Algorithms & Data Structures
- Why are logarithms useful
- The History of the Natural Logarithm - How was it discovered?
Websites about Exponential Growth
- Exponential growth 2: real-life lessons from the COVID-19 pandemic
- https://www.dw.com/en/exponential-growth-makes-india-the-global-covid-hot-spot/a-57364927
Probability distributions
Entropy
Huffman coding
- Huffman Coding Implementation in Python with Example
- Huffman Encoding — Compression basics in Python
- Huffman coding in Python
- Huffman Coding
- Huffman Coding (Wikipedia)
- Is there a way to guarantee hierarchical output from NetworkX?
Python
- Stripping everything but alphanumeric chars from a string in Python
- Python regex
- Python's Counter: The Pythonic Way to Count Objects
- (NumPy random sampling)[https://numpy.org/doc/stable/reference/random/index.html]
- (Python string formatting)[https://www.w3schools.com/python/python_string_formatting.asp]
- (Compute Bigrams Frequency in a String in Python)[https://www.codespeedy.com/compute-bigrams-frequency-in-a-string-in-python/]
Jupyter Lite * Markdown
Matplotlib
- Plot mathematical functions
- Matplotlib grid
- Pie charts
- Save Matplotlib Plot with Transparent Background
- Creating annotated heatmaps
- Better Heatmaps and Correlation Matrix Plots in Python
- Discrete distribution as horizontal bar chart
- my matplotlib title gets cropped
- Tight Layout guide