image-processing-handbook.github.io
A working reference for digital image processing, written for people who ship image code rather than for people who only read about it.
Image processing sits in an awkward place. The mathematics is old and settled, the libraries are excellent, and yet almost every project runs into the same handful of problems. A resize that softens detail. A filter that crawls on large inputs. A colour shift nobody can explain.
This handbook is written for those moments. Nine modules move from the physics of capture to the neural models that now sit at the end of many pipelines, and they keep returning to one question. What is actually happening to the numbers?
Anyone who has to make images behave: web developers building editors and upload pipelines, graphics programmers writing shaders, and engineers who inherited a processing chain and need to understand why it produces what it produces.
It assumes you can read code and are comfortable with arrays and loops. It does not assume a signal processing background. Where mathematics is unavoidable it is written out and explained rather than cited.
Every section follows the same rhythm. The idea in plain language, the operation written out, a figure, and then the failure modes.
The failure modes are the part most references leave out. Textbooks show the clean case, and production images are rarely clean. So the handbook spends its time on the boundary that produces a dark halo, the colour space that quietly darkens a downscale, the sharpening that looks crisp at thumbnail size and wrong at full size.
Code samples are plain JavaScript with typed arrays, no framework, no library version to go stale. The naive implementation comes first because it is the one you can read. Optimised variants follow, and the difference between them is usually memory access order rather than arithmetic.
| Module | What it covers |
|---|---|
| Getting Started | Scope, notation, coordinate order, and the value ranges every later module assumes. |
| 01. Foundations of Digital Images | Sampling and quantisation, channels and bit depth, colour spaces, and how pixels sit in memory. |
| 02. Point Operations and Histograms | Per-pixel mappings, lookup tables, equalisation and CLAHE, and how to choose a threshold. |
| 03. Spatial Filtering and Convolution | Kernels and boundary handling, Gaussian and median and bilateral, Sobel and Canny, unsharp masking. |
| 04. Geometry and Interpolation | Affine and perspective mapping, nearest through Lanczos, and the quality against cost trade-off. |
| 05. Frequency Domain Processing | The Fourier transform, amplitude against phase, and the periodic noise that only a notch filter kills. |
| 06. Morphology and Contours | Erosion and dilation, opening and closing, skeletons, contour hierarchies, watershed segmentation. |
| 07. Compression and Formats | Entropy coding, the DCT and JPEG, WebP and AVIF, and palette quantisation. |
| 08. High-Performance Web Architecture | Typed arrays and cache behaviour, fragment and compute shaders, workers, WebAssembly. |
| 09. Neural Methods | Where learned models replace hand-written kernels: super-resolution, segmentation, inpainting. |
The ordering is deliberate. Later modules assume the vocabulary of earlier ones, so a full pass in order is the fastest route to fluency.
Nothing stops you opening Module 05 on a Tuesday afternoon because a periodic pattern turned up in a scan. Each module stands on its own, and cross-references point back to whatever it depends on.
Three habits the handbook argues for throughout:
- Read the operation before the code. Most bugs in image pipelines are conceptual. A wrong colour space will not look like a bug, it will look like a slightly disappointing result.
- Reproduce on a small array. A five by five test image with hand-written values exposes boundary and rounding errors that a photograph hides completely.
- Measure before optimising. Reach for the profiler after the correct result exists, not before.
The structure is complete and every module is published. Depth varies: some sections are finished, others are still closer to an outline than to the treatment they deserve.
Figures are being produced. Each one carries the brief it was written from, so any illustration can be regenerated or replaced without guessing at the original intent. Open the panel under a figure to read it.
Corrections and expansions are welcome, particularly failure modes met in real projects. See CONTRIBUTING.md for how the site is built and how to add or edit a page.