Encoder–Decoder Architecture

A complete, hands-on teaching package for the 120-minute live session "Encoder–Decoder Architecture", built for starters of machine learning.

Everything runs in Google Colab with zero installs. Keras 3 / TensorFlow.

The one takeaway: The encoder is paid to forget WHERE. The decoder is asked to rebuild it. Everything that goes wrong — and U-Net — follows from that one sentence.

---

The idea behind the design

Most encoder–decoder material teaches a recipe: compress, then reconstruct; add skip connections to make it sharper. Learners can repeat it and cannot reason with it.

This package teaches a causal chain instead, and demonstrates every link:

  max-pooling destroys WHERE
             ↓
  the bottleneck cannot afford WHERE
             ↓
  the decoder is uncertain about WHERE
             ↓
  squared error tells it to hedge
             ↓
            BLUR

Once a learner believes that chain, U-Net stops being a clever trick and becomes the only sensible response — the encoder still had those positions, so hand them over. And the session ends by showing what that costs, so learners can choose rather than reflexively add skips.

Pedagogy is WHY → WHAT → HOW throughout, with one recurring question asked at every architectural decision:

"What information just got destroyed here — and can the decoder get it back?"

---

What's in the box

encoder-decoder-2/
├── notebooks/                        6 standalone Colab notebooks, pre-run with outputs
│   ├── 01_the_copy_machine_paradox.ipynb        (15 min · conceptual)
│   ├── 02_cnn_encoder_what_vs_where.ipynb       (20 min · concept + demo)
│   ├── 03_the_bottleneck.ipynb                  (15 min · concept + demo)
│   ├── 04_why_reconstruction_blurs.ipynb        (10 min · conceptual, no training)
│   ├── 05_build_encoder_decoder.ipynb           (35 min · guided build)
│   └── 06_unet_skip_connections.ipynb           (25 min · guided build)
├── slides/
│   └── encoder_decoder.html          reveal.js deck, speaker notes on every slide (press S)
├── teaching/
│   ├── instructor_guide.md           minute-by-minute run sheet, Q&A bank, contingencies
│   ├── learner_handout.md            take-home notes, shape cheat-sheet, glossary
│   ├── analogies.md                  the analogy bank — delivery scripts + where each breaks
│   ├── exercises.md                  graded exercises + a segmentation capstone
│   └── solutions/solutions.md        worked solutions, with measured numbers
├── data/results/                     every measured number, as JSON
├── scripts/
│   ├── run_benchmarks.py             reproduces every number on the slides
│   ├── run_exercise_benchmarks.py    reproduces every number in the solutions
│   ├── build_notebooks.py            regenerates the notebooks from one source of truth
│   ├── nbtools.py                    notebook-generation helpers + the shared cells
│   └── nb_sources/                   the six notebooks, as editable Python
└── docs/superpowers/specs/           the design spec for this package

---

For learners — open a notebook and go

#	Notebook		Min
01	The copy-machine paradox — why compress at all		15
02	The CNN encoder — WHAT vs WHERE		20
03	The bottleneck and the latent space		15
04	Why reconstruction blurs		10
05	Build it — encoder, bottleneck, decoder		35
06	U-Net — handing WHERE back		25

Every notebook also carries its own Open in Colab badge at the top.

Open a notebook, run the SETUP cell, go. That is the whole setup.

• No pip install — Colab already ships Keras 3, TensorFlow, NumPy and Matplotlib.
• No repo to clone. Each notebook is fully standalone; helpers are duplicated into a single SETUP cell on purpose. In a live session, one broken import blocks thirty people.
• Every notebook ships pre-run with its outputs visible, so you can read it without running anything.
• Fashion-MNIST (~30 MB) downloads inside the SETUP cell in a few seconds.

Compute: every model trains in 60–90 s on a laptop CPU, or roughly 1.5–3 min on a free Colab CPU. No GPU needed. If your hardware is slow, change epochs=10 to epochs=5 — the story is identical and every relationship holds.

---

For instructors — start here

1. Read teaching/instructor_guide.md §3, the minute-by-minute run sheet. That is the spine.
2. Open slides/encoder_decoder.html and press S for speaker notes — every slide has them, including the exact numbers to quote.
3. Skim teaching/analogies.md for the delivery script of each analogy, and — importantly — where each one breaks down.
4. Before the room fills, run the SETUP cell in all six notebooks.

The three things that must not be cut:

#	The moment	Why
1	The three PREDICT checkpoints	Being wrong with skin in the game is what makes the correction stick
2	The error map in notebook 05	The WHAT/WHERE split becomes visible in a model they built
3	The plain vs U-Net A/B in notebook 06	The payoff, and the trade-off that follows it

---

The measured results

Fashion-MNIST · 12 000 train / 2 000 test · 10 epochs · batch 256 · Adam · MSE · seed 42 TensorFlow 2.21.0 / Keras 3.15.1

latent_dim	compression	plain PSNR	U-Net PSNR	plain detail	U-Net detail
128	6×	19.49 dB	33.63 dB	51.7%	97.8%
32	25×	18.50 dB	33.99 dB	45.9%	98.1%
8	98×	17.18 dB	31.55 dB	36.6%	98.8%
2	392×	14.65 dB	34.61 dB	24.8%	98.0%

The plain column collapses as the bottleneck shrinks. The U-Net column does not even decline — it wanders a couple of dB with no relationship to bottleneck size at all, and a U-Net with 2 numbers beats a plain autoencoder with 128.

That is the session's punchline: in a U-Net, most information never passes through the bottleneck. It travels through the skips — which is exactly why skips are right for segmentation and wrong for representation learning.

Reproducibility

Run the notebooks top-to-bottom on a fresh runtime and they reproduce the table above exactly — notebook 03 gives 19.49 and 17.18 dB, notebook 06 gives 17.18 and 31.55 dB. Every model is constructed immediately after set_random_seed(42) precisely so this holds.

The one way to break it: re-running a training cell without re-running the build cell above it. TensorFlow draws starting weights at construction time, and its random stream depends on how many operations the process has already created, so a second build_model() call in the same session does not start where the first one did.

That failure mode is informative in itself: the plain model barely moves, while the U-Net shifts by a few dB. The plain model is tightly constrained — everything passes through 8 numbers, so there is essentially one good solution. The U-Net, with skips carrying the picture, has many near-equivalent ones. So when comparing models, read the relationships rather than the third decimal: ~+15 dB over plain, ~98% detail retained, no dependence on bottleneck size.

python scripts/verify_claims.py    # checks every number in the docs against data/results/

---

Reproducing everything

python -m venv .venv && source .venv/bin/activate
pip install "tensorflow>=2.18" numpy matplotlib jupyter nbconvert

python scripts/run_benchmarks.py           # every number on the slides      (~15 min, CPU)
python scripts/run_exercise_benchmarks.py  # every number in the solutions   (~10 min, CPU)
python scripts/build_notebooks.py          # regenerate the six notebooks

Note on the workflow: build_notebooks.py regenerates notebooks from scripts/nb_sources/ and therefore clears their stored outputs. To ship them pre-run, execute them afterwards:

for f in notebooks/*.ipynb; do
  python -m nbconvert --to notebook --execute --inplace --ExecutePreprocessor.timeout=2400 "$f"
done

Measured tables are injected into the notebooks at build time from data/results/, so no number in this package was ever typed by hand.

---

Design decisions, and why

Decision	Why
Fashion-MNIST, not MNIST	Stripes, laces and buttons are fine-grained WHERE — the first thing a bottleneck kills and the most visibly restored by skips. A blurry 7 still reads as a 7, which quietly undermines the lesson.
Reconstruction, not segmentation	Makes the A/B fair and fast, and reconstruction is the cleanest possible test of "did the code keep enough?". Segmentation is the capstone exercise.
UpSampling2D + Conv2D, not Conv2DTranspose	One fewer artifact (checkerboarding) to explain while learners are still working out what the architecture does. The transpose version is an exercise.
One builder, one flag for the A/B	Any measured difference can only be caused by the skips. Building a fair A/B is a skill in itself and most people never learn it.
A detail metric alongside PSNR	PSNR falls gently while the pictures fall apart. Teaching that a metric can be blind to the failure you care about is worth 60 seconds of class time.
Standalone notebooks, duplicated helpers	DRY is the wrong optimisation for a live classroom. One broken import blocks the room.

---

Not covered

Mentioned in one closing slide as "where this goes next", not taught: VAEs and probabilistic latents, seq2seq / transformer encoder–decoders, attention, perceptual and adversarial losses, and diffusion. The capstone exercise extends the U-Net to segmentation.