Inference Systems

Engineering Generative AI from Kernel to Cluster

This repository contains an original, open technical book about modern generative-model inference. It develops systems principles from primary research and production implementations, including vLLM and SGLang.

Read the published book at igniting.github.io/inference. The manuscript contains 27 chapters across five parts, plus practical appendices, a glossary, a source ledger, and a complete worked-solutions manual. Its research and originality rules are documented in the research policy.

Every chapter includes at least two block diagrams and a compact comparison table. A custom Dagre-backed SVG renderer keeps block geometry, typography, connectors, and responsive reflow consistent while the diagram source remains versioned beside the prose. A single signal-blue visual system unifies the prose, navigation, code, diagrams, and tables.

Read locally

The book uses mdBook.

mdbook serve --open

To verify the static build:

mdbook build

The GitHub Actions workflow publishes every successful main build to GitHub Pages.

Status

• Define the thesis, audience, and boundaries
• Establish an evidence and originality policy
• Draft all 27 chapters and nine appendices
• Review coverage and pedagogy across the complete manuscript
• Revise the complete manuscript after editorial critique
• Replace open-ended exercises with worked examples and full solutions
• Add a custom professional mdBook theme
• Publish reproducibly through GitHub Pages

The source ledger pins the implementation revisions studied for this edition. Performance claims should be accompanied by enough workload, hardware, and software detail to reproduce them.