August 2026 Release

This release expands the bilingual deep-learning curriculum with new and refined material on loss functions, neural-network trainability, convolutional networks, VAEs, diffusion models, attention, MLPs, memory engineering, and LLM training engineering. It also adds a complete Stanford CS336 Assignment 1 implementation, substantially improves dnnlpy, and modernizes the project’s build, rendering, and release infrastructure.

New Notebooks

Chapter 1: Introduction to Deep Learning

• 1.2 Loss Function: How Does a Model Know How Wrong It Is?
• 1.3 Forward Propagation, Backpropagation, and Computation Graph [rewrite]
• 1.5 Why Neural Networks Can Be Trained: Optimization Intuition in High-Dimensional Spaces

Chapter 5: Convolutional Neural Networks

• 5.1 From MLP to CNN: Why Images Need Convolution
• 5.2 Convolution: Kernels, Padding, Stride, and Channels
• 5.3 Implementing Conv2d from Scratch: From Sliding Windows to PyTorch Modules
• 5.4 Pooling and Downsampling: Max, Average, and Adaptive Pooling
• 5.5 Building a Simple CNN: From Feature Extraction to Image Classification
• 5.6 LeNet: An Early Template for Convolution, Pooling, and Fully Connected Layers

Chapter 14: Diffusion Models: From Denoising to Generation

• 14.1 DDPM: From Denoising to Generation [rewrite]

Chapter 19: LLM Training Engineering

• 19.1 Where Does LLM Training Memory Go? Model States, Activations, and Runtime Overhead

Stanford CS336 Assignment 1

• Added a complete Transformer language-model implementation for Assignment 1.
• Added linear layers, embeddings, RMSNorm, SwiGLU, Rotary Position Embeddings, multi-head self-attention, Transformer blocks, and a Transformer language model.
• Added data loading, cross-entropy, gradient clipping, AdamW, checkpoint serialization, and learning-rate scheduling utilities.
• Added a compatible BPE tokenizer with byte-level processing, special-token support, encoding, decoding, and training.
• Added an optimized incremental BPE training implementation.
• Expanded the assignment write-up with profiling results, implementation analysis, and Mermaid diagrams.
• Added comprehensive reference fixtures, snapshots, and tests derived from the assignment interface.

dnnlpy Package Updates

• Added one-, two-, and three-dimensional max pooling and average pooling modules.
• Added adaptive max pooling and adaptive average pooling APIs.
• Added matching functional pooling implementations and expanded public exports.
• Added device-memory inspection and cleanup utilities.
• Added reusable byte-conversion utilities.
• Added CS336 Assignment 1 model and utility modules.
• Improved traditional tokenizer training with persistent model state.
• Improved tokenizer compatibility across supported Python and dependency versions.
• Aligned BPE tokenization behavior more closely with Hugging Face tokenizers.
• Refined tokenizer vocabulary, merge, batch-encoding, and special-token handling.
• Improved Trainer metric collection, evaluation history, logging, and state tracking.
• Modernized model, optimizer, activation, folding, convolution, and gradient-clipping APIs.
• Renamed the linear-layer parameter attributes to weight and bias and aligned the MLP tests with the public API.
• Expanded tests for pooling, tokenizers, models, optimizers, training utilities, and public exports.
• Adjusted the TinyStories tokenizer performance test for more reliable execution across environments.
• Updated project metadata and dependencies for PyTorch 2.13 and Transformers 5.15.
• Marked dnnlpy as stable and finalized its version as 2026.08.24.

Book and Documentation Updates

• Renumbered the Attention and Transformer material from Chapter 8 to Chapter 9.
• Removed the incomplete GAN and vision-language-model chapters from the current book structure.
• Revised foundational PyTorch, MLP, optimization, efficient-attention, Vision Transformer, VAE, diffusion, regularization, and GPT-2 material.
• Refreshed bilingual diagrams and standardized Mermaid source and SVG naming.
• Updated installation instructions, repository links, contribution guidance, citation metadata, and project roadmaps.
• Added an AI-assisted-writing disclosure, book-author and website metadata, and cover-design attribution.
• Improved the CS336 notebook references and generated notebook artifacts.
• Added a new book cover and refreshed Typst styling for code, quotations, and bilingual PDF output.
• Added dedicated Typst reference-section handling and corrected part-title sizing and table-header alignment in the PDF output.
• Added the missing English chapters and aligned them with the Chinese version.

Build and Development Updates

• Added a Dockerfile and development-container configuration.
• Added pre-commit configuration and refreshed Ruff and Python tooling.
• Added Markdown linting configuration and expanded Ruff formatting coverage for Markdown, Quarto, and notebook files.
• Updated Quarto profiles for HTML, notebook, and bilingual Typst rendering.
• Bumped the minimum Quarto version to 1.10.
• Added Posit Connect Cloud publishing configuration.
• Simplified cache, checkpoint, attachment, image-attribute, table-of-contents, and PDF-renaming utilities.
• Removed committed Quarto freeze artifacts and consolidated cache cleanup.
• Updated dataset-download and Mermaid-cleanup tooling.
• Finalized the project and dnnlpy versions as 2026.08.24 and marked the release metadata as stable.

CI Updates

• Consolidated website, notebook, PDF, and pull-request rendering workflows.
• Added manually dispatched website and bilingual PDF rendering workflows.
• Added Docker image release automation.
• Improved platform-aware execution, caching, artifact packaging, and release handling.
• Standardized workflow concurrency behavior and restored published-release triggers for package and Docker publishing.
• Restricted PyPI, TestPyPI, and notebook-sync side effects to the canonical jshn9515/deep-learning-notes repository.
• Made matrix virtual-environment creation independent of the root project’s Python requirement.
• Updated GitHub Actions dependencies, including newer Python, uv, checkout, and cache actions.
• Simplified dnnlpy testing and packaging automation.

Merged Pull Requests

• DEP: Update dependency transformers to >=5.14.1,<5.15.0 by @renovate[bot] in #2
• DEP: Update dependency transformers to >=5.15.0,<5.16.0 by @renovate[bot] in #6
• DEP: Update astral-sh/setup-uv action to v10 by @renovate[bot] in #7
• DEP: Update dependency tiktoken to >=0.14.0,<0.15.0 by @renovate[bot] in #9
• [zh] FIX: Correct InstanceNorm example input usage by @wqpwqp1222 in #11
• MNT: Simplify GroupNorm statistics example by @wqpwqp1222 in #12
• DEP: Update dependency diffusers to >=0.40.0,<0.41.0 by @renovate[bot] in #13
• DOC: Improve the wording and clarity of the tutorial content by @tiansiyuan in #65
• [zh] FIX: Correct conv2d output size example variable name by @wqpwqp1222 in #69
• [zh] DOC: Clarify Conv2d padding and fix math formatting by @wqpwqp1222 in #70

New Contributors

• @renovate[bot] made their first contribution in #2
• @wqpwqp1222 made their first contribution in #11
• @tiansiyuan made their first contribution in #65

Note

This project continues to be maintained in both Chinese and English through a Quarto-based structure, as an open and continuously growing collection of deep learning study notes.

Full Changelog: v2026.08.21-rc1...v2026.08.24

August 2026 Pre-release

This release expands the bilingual deep-learning curriculum with new material on loss functions, neural-network trainability, convolutional networks, and LLM training engineering. It also adds a complete Stanford CS336 Assignment 1 implementation, substantially improves dnnlpy, and modernizes the project’s build, rendering, and release infrastructure.

New Notebooks

Chapter 1: Introduction to Deep Learning

• 1.2 Loss Function: How Does a Model Know How Wrong It Is?
• 1.3 Forward Propagation, Backpropagation, and Computation Graph [rewrite]
• 1.5 Why Neural Networks Can Be Trained: Optimization Intuition in High-Dimensional Spaces

Chapter 5: Convolutional Neural Networks

• 5.1 From MLP to CNN: Why Images Need Convolution
• 5.2 Convolution: Kernels, Padding, Stride, and Channels
• 5.3 Implementing Conv2d from Scratch: From Sliding Windows to PyTorch Modules
• 5.4 Pooling and Downsampling: Max, Average, and Adaptive Pooling
• 5.5 Building a Simple CNN: From Feature Extraction to Image Classification
• 5.6 LeNet: An Early Template for Convolution, Pooling, and Fully Connected Layers

Chapter 19: LLM Training Engineering

• 19.1 Where Does LLM Training Memory Go? Model States, Activations, and Runtime Overhead

Stanford CS336 Assignment 1

• Added a complete Transformer language-model implementation for Assignment 1.
• Added linear layers, embeddings, RMSNorm, SwiGLU, Rotary Position Embeddings, multi-head self-attention, Transformer blocks, and a Transformer language model.
• Added data loading, cross-entropy, gradient clipping, AdamW, checkpoint serialization, and learning-rate scheduling utilities.
• Added a compatible BPE tokenizer with byte-level processing, special-token support, encoding, decoding, and training.
• Added an optimized incremental BPE training implementation.
• Expanded the assignment write-up with profiling results, implementation analysis, and Mermaid diagrams.
• Added comprehensive reference fixtures, snapshots, and tests derived from the assignment interface.

dnnlpy Package Updates

• Added one-, two-, and three-dimensional max pooling and average pooling modules.
• Added adaptive max pooling and adaptive average pooling APIs.
• Added matching functional pooling implementations and expanded public exports.
• Added device-memory inspection and cleanup utilities.
• Added reusable byte-conversion utilities.
• Added CS336 Assignment 1 model and utility modules.
• Improved traditional tokenizer training with persistent model state.
• Improved tokenizer compatibility across supported Python and dependency versions.
• Aligned BPE tokenization behavior more closely with Hugging Face tokenizers.
• Refined tokenizer vocabulary, merge, batch-encoding, and special-token handling.
• Improved Trainer metric collection, evaluation history, logging, and state tracking.
• Modernized model, optimizer, activation, folding, convolution, and gradient-clipping APIs.
• Expanded tests for pooling, tokenizers, models, optimizers, training utilities, and public exports.
• Updated project metadata and dependencies for PyTorch 2.13 and Transformers 5.15.

Book and Documentation Updates

• Renumbered the Attention and Transformer material from Chapter 8 to Chapter 9.
• Removed the incomplete GAN and vision-language-model chapters from the current book structure.
• Revised foundational PyTorch, MLP, optimization, efficient-attention, Vision Transformer, VAE, diffusion, regularization, and GPT-2 material.
• Refreshed bilingual diagrams and standardized Mermaid source and SVG naming.
• Updated installation instructions, repository links, contribution guidance, citation metadata, and project roadmaps.
• Improved the CS336 notebook references and generated notebook artifacts.
• Added a new book cover and refreshed Typst styling for code, quotations, and bilingual PDF output.
• Added the missing English chapters and aligned them with the Chinese version.

Build and Development Updates

• Added a Dockerfile and development-container configuration.
• Added pre-commit configuration and refreshed Ruff and Python tooling.
• Updated Quarto profiles for HTML, notebook, and bilingual Typst rendering.
• Bumped the minimum Quarto version to 1.10.
• Simplified cache, checkpoint, attachment, image-attribute, table-of-contents, and PDF-renaming utilities.
• Removed committed Quarto freeze artifacts and consolidated cache cleanup.
• Updated dataset-download and Mermaid-cleanup tooling.

CI Updates

• Consolidated website, notebook, PDF, and pull-request rendering workflows.
• Added manually dispatched website and bilingual PDF rendering workflows.
• Added Docker image release automation.
• Improved platform-aware execution, caching, artifact packaging, and release handling.
• Updated GitHub Actions dependencies, including newer Python, uv, checkout, and cache actions.
• Simplified dnnlpy testing and packaging automation.

Merged Pull Requests

• DEP: Update dependency transformers to >=5.14.1,<5.15.0 by @renovate[bot] in #2
• DEP: Update dependency transformers to >=5.15.0,<5.16.0 by @renovate[bot] in #6
• DEP: Update astral-sh/setup-uv action to v10 by @renovate[bot] in #7
• DEP: Update dependency tiktoken to >=0.14.0,<0.15.0 by @renovate[bot] in #9
• [zh] FIX: Correct InstanceNorm example input usage by @wqpwqp1222 in #11
• MNT: Simplify GroupNorm statistics example by @wqpwqp1222 in #12
• DEP: Update dependency diffusers to >=0.40.0,<0.41.0 by @renovate[bot] in #13
• DOC: Improve the wording and clarity of the tutorial content by @tiansiyuan in #65
• [zh] FIX: Correct conv2d output size example variable name by @wqpwqp1222 in #69
• [zh] DOC: Clarify Conv2d padding and fix math formatting by @wqpwqp1222 in #70

New Contributors

• @renovate[bot] made their first contribution in #2
• @wqpwqp1222 made their first contribution in #11
• @tiansiyuan made their first contribution in #65

Note

This project continues to be maintained in both Chinese and English through a Quarto-based structure, as an open and continuously growing collection of deep learning study notes.

Full Changelog: https://github.com/datawhale/deep-learning-notes/commits/v2026.08.20-rc1