Image files

Image files

render/png.mojo, render/jpeg.mojo, render/tga.mojo, render/rgbe.mojo, render/exr.mojo, render/inflate.mojo, render/apng.mojo, render/ppm.mojo, render/tiff.mojo and render/webp.mojo. The project writes PNG, APNG and PPM, and reads PNG, JPEG, WebP, TGA, TIFF, Radiance HDR and OpenEXR. No compression or image library is involved.

three.js: TextureLoader for reading PNG, JPEG and WebP, TGALoader for TGA, and TIFFLoader for TIFF. three.js writes nothing; the browser does.

Formats

Format	Read	Write	Alpha	Note
PNG	Yes	Yes	8-bit	The default output.
APNG	No	Yes	8-bit	Many frames in one file. The first frame is a plain PNG.
PPM	No	Yes	No	Plain text, for reading pixel values in an editor.
JPEG	Yes	No	No	Baseline and progressive. See Read a JPEG.
WebP	Yes	No	8-bit	Lossy and lossless. See Read a WebP.
TGA	Yes	No	8-bit	See Read a TGA.
TIFF	Yes	No	8-bit	Baseline TIFF. See Read a TIFF.
Radiance HDR	Yes	No	No	Linear floats. See HDR images.
OpenEXR	Yes	No	Float	Linear floats. See HDR images.

Write a PNG

from render.png import encode
Path("out/frame.png").write_bytes(encode(framebuffer))

The encoder uses DEFLATE's stored blocks, so the file is larger than a compressed one and legal everywhere.

Write an APNG

from render.apng import encode
Path("out/turn.png").write_bytes(encode(frames, delay_ms=60))

frames is a List[Framebuffer] of one size. The encoder raises for an empty list or a frame of a different size.

Recompress a PNG

tools/optimize_png.mojo makes a PNG or APNG file smaller without a change to its pixels. Use it after you write an image.

mojo run -I . tools/optimize_png.mojo out/NAME.png

The tool decodes each image stream and encodes it again at DEFLATE level 9. A stream is a run of IDAT chunks, or a run of fdAT chunks after one fcTL chunk. The new stream goes into the first chunk of the run. The other chunks of the run become empty.

The tool keeps the position of each chunk. The metadata, frame delays, blend and disposal operations, APNG sequence numbers and filtered scanlines do not change. The tool replaces the file only when the new file is smaller. It writes a temporary file, gives it the mode of the old file, and renames it over the old file.

Each gallery recipe runs the tool after it renders. make optimize-images recompresses all tracked gallery images. The library half is render/png_optimize.mojo.

Write a PPM

from render.ppm import to_stdout
to_stdout(framebuffer)

Read a PNG

from render.png import decode
from render.texture import texture_from

var image = decode(Path("assets/brick.png").read_bytes())
var skin = assets.textures.add(texture_from(image, REPEAT, BILINEAR, mipmapped=True))

decode returns a DecodedImage: width, height, RGBA pixels, and a color_space.

Supported	Refused by name
Grayscale, RGB, palette, grayscale with alpha, RGBA, all at 8 bits	16-bit channels
A tRNS transparency chunk	Palettes below 8 bits
Every row filter	Adam7 interlacing
Stored, fixed-Huffman and dynamic-Huffman DEFLATE blocks

Read a JPEG

from render.jpeg import decode
from render.texture import texture_from

var photo = decode(Path("assets/photo.jpg").read_bytes())
var skin = assets.textures.add(texture_from(photo, REPEAT, BILINEAR, mipmapped=True))

render/jpeg.mojo. decode returns a DecodedImage, as the PNG reader does, always SRGB: a JPEG declares no color space, and every viewer assumes that one. No image library is involved.

Supported	Refused by name
Baseline sequential Huffman coding, SOF0 and SOF1 at 8 bits	Arithmetic coding
Progressive Huffman coding, SOF2: spectral selection and successive approximation	Twelve bits a sample, and sixteen-bit quantization tables
Gray, and YCbCr in three components numbered one, two and three as JFIF numbers them	Four components, which is CMYK, and three numbered any other way
4:4:4, 4:2:2 and 4:2:0	Other sampling factors
Restart intervals, in sequential and progressive scans
A scan of every component, or a scan per component
Application and comment segments, skipped

Progressive files

A progressive file sends the same quantized coefficients as a baseline file, in pieces. The decoder reads every scan into one store of coefficients. It transforms the store once, after the end marker. A baseline file goes through the same store in one scan. A progressive file and its baseline twin at the same quality decode to the same samples, bit for bit. tests/test_jpeg.mojo checks this for 4:2:0, 4:2:2, 4:4:4, gray and restart intervals.

The decoder reads the four kinds of progressive scan:

Scan	What it sends
DC first	The high bits of each block's DC coefficient, as a difference from the last block's.
DC refinement	One more bit of each DC coefficient, raw.
AC first	The high bits of a band of AC coefficients of one component, with end-of-band runs that span blocks.
AC refinement	One more bit of a band: a correction bit for each nonzero coefficient, and new coefficients of one bit.

The decoder checks the order of the scans. A coefficient's first bits must come before its refinements, and each refinement must add exactly one bit. The AC scans of a component must come after its first DC scan. Every component must be in some scan. A file that breaks one of these rules is refused.

What the decoder computes

The inverse cosine transform is the separable float one, the standard's own definition, rounded once at the end. A chroma component stored at half size is read through a triangle filter. The standard leaves the reconstruction filter to the decoder, and this one uses the filter libjpeg calls fancy upsampling so that the two agree. At the right and bottom edges, the filter repeats the component's last real sample, as libjpeg does. It does not read the encoder's padding.

Two decoders agree on a JPEG to within a level or two and never to the bit. tests/test_jpeg.mojo holds this one to two levels of libjpeg's output. One checkered test picture with sharp chroma edges is held to three levels.

Speed

A 2048 by 2048 color JPEG decodes in about 0.37 seconds, and a gray one in about 0.13 seconds. Three things make it fast:

• The bit reader reads whole bytes ahead into a buffer of up to 56 bits. It stops at a marker, so a bit past the end of a scan is still refused.
• Each Huffman table has a lookup of 512 entries. One look at the next nine bits gives the symbol of every code of nine bits or fewer. A longer code is read a bit at a time.
• The inverse transform works in stack arrays, one block after another. Each chroma plane is upsampled once for the whole image.

The pixels are the same as before, bit for bit.

Integrity checks

The decoder checks that every marker segment fits and that the frame header comes once and before the scans. It checks that every table a scan names was defined and holds no zero, and that each scan names components the frame has. It checks that the restart markers arrive in order and that the end marker closes the file. A truncated scan, a code no table defines and a run past a block's end are refused rather than padded.

Read a TGA

from render.tga import decode
from render.texture import texture_from

var image = decode(Path("assets/skin.tga").read_bytes())
var skin = assets.textures.add(texture_from(image, REPEAT, BILINEAR))

render/tga.mojo. decode returns a DecodedImage, as the PNG and JPEG readers do, always SRGB. The rows run from the top down, whichever corner the file starts at. The reader follows three.js's TGALoader.

Image type	Pixel sizes	Note
Color-mapped, 1, and run-length color-mapped, 9	8-bit indices	The map holds at most 256 entries of 24 bits.
True color, 2, and run-length true color, 10	24 and 32 bits	A 32-bit pixel carries its alpha.
Gray, 3, and run-length gray, 11	8 and 16 bits	A 16-bit pixel is a level and an alpha.

The descriptor byte gives the origin. The reader reads all four corners: bottom left, bottom right, top left and top right. It does not read the attribute bits, and three.js does not read them either. It ignores the image id and the TGA 2.0 footer.

Where the TGA reader differs from three.js

• The reader refuses 16-bit true color by name. three.js reads the attribute bit as alpha with the opposite meaning from what writers use.
• The reader applies the color map's first entry index. three.js ignores it.
• The reader refuses an index past the color map, a packet past the image and a file that ends early. three.js reads these as whatever its array holds.

decode_image in loaders/gltf.mojo reads a TGA when the bytes are neither PNG nor JPEG. A TGA has no signature, so the header check alone tells a TGA from other bytes. glTF names only PNG and JPEG, so this fallback is an addition of this port.

Read a TIFF

from render.tiff import read_tiff

var scan = assets.textures.add(read_tiff(Path("assets/tiff/rgb_lzw.tif").read_bytes()))

render/tiff.mojo. read_tiff reads the first image of a TIFF file into a texture, as three.js's TIFFLoader makes it. three.js reads the file with UTIF. This decoder follows UTIF's arithmetic for baseline TIFF. The texture is LINEAR, filtered linearly with a chain of levels, and flipped.

Part	Read
Byte order	II and MM. A 16-bit sample of either order.
Layout	Strips and tiles.
Compression	None, LZW, Deflate and PackBits, with or without the horizontal predictor.
White is zero, 0	1, 4, 8 and 16-bit gray.
Black is zero, 1	1, 2, 8 and 16-bit gray, and 32-bit float gray.
RGB, 2	8-bit with one, three, or four or more samples. 16-bit with three or four samples. 32-bit floats with three or four samples.
Palette, 3	1, 2, 4 and 8-bit indices. An extra sample is the alpha.
CMYK, 5	UTIF's formula. A fifth sample is the alpha.

A 16-bit sample keeps its high byte. A 32-bit float RGB sample is clamped to zero to one and encoded with sRGB's curve, through UTIF's table of 65536 steps.

Same as three.js

The decoder keeps these UTIF behaviors:

• 4-bit black-is-zero gray, 2-bit white-is-zero gray, and RGB of other sample counts are transparent black.
• A float RGB image with a negative sample is read in the other byte order, as UTIF guesses.
• CMYK with four samples reads a fifth sample anyway, times zero. Past the end of the data that is NaN, so the last pixel is transparent.
• An uncompressed image of one strip is read as long as the image, whatever its byte count says. A Deflate strip that does not fit is dropped.
• An LZW code past the table is read as the next code.

Errors

The decoder refuses these, with a message that names the problem:

• What three.js throws on. In three.js's build of UTIF, each log is a throw. So an unknown compression, an unknown photometric form, planar configuration 2 and an unknown tag type throw, as does 32-bit gray that is not float.
• What this decoder does not read: CCITT, JPEG, YCbCr, and the camera and raw formats.
• An LZW code past the table right after a clear code. UTIF reads it from a table that it shares between decodes, so its output depends on what it decoded before.
• Strips that cover fewer rows than the image. UTIF cuts the data short, and each photometric form reads the missing bytes in its own way.
• An image of no width or no height, and a file that starts with neither II nor MM. Also a tile with no offset or no byte count. three.js gives an empty image for the first two. For such a tile, UTIF reads what it can, differently for each compression.

The tests read 96 files that assets/tiff/make_tiff.py wrote, with Pillow and by hand. Each one must have the bytes that three.js 0.180 gives, or be one of the seven that the decoder refuses on purpose.

Read a WebP

from render.webp import decode
from render.texture import texture_from

var image = decode(Path("assets/webp/lossy_q75.webp").read_bytes())
var photo = assets.textures.add(texture_from(image, REPEAT, BILINEAR))

render/webp.mojo, render/webp_lossless.mojo and render/webp_lossy.mojo. decode returns a DecodedImage, eight-bit RGBA from the top, always SRGB. In three.js, the browser decodes WebP with libwebp. This decoder is a port of libwebp 1.6.0, and its output is the same bytes as libwebp's dwebp.

Part	Read
Container	A RIFF WEBP file, with or without a VP8X chunk. Other chunks, such as metadata, are skipped.
Lossless, VP8L	The predictor, cross-color, subtract-green and color-indexing transforms, the color cache, and groups of Huffman codes.
Lossy, VP8	A VP8 key frame with segments, one to eight token partitions, and the simple or normal loop filter.
Alpha, ALPH	Raw or lossless, with no filter or a horizontal, vertical or gradient filter.

A lossy image becomes RGB as libwebp makes it by default: fancy upsampling of the chroma, and libwebp's fixed-point conversion. The last ALPH chunk counts, as in libwebp.

Errors

The decoder refuses these:

• What libwebp refuses. For example, a chunk or a RIFF size that runs past the file, a canvas that is not the image's size, or a malformed stream. Also alpha that does not decode.
• An animation. libwebp's still-image decoder refuses it too.
• A bare VP8 or VP8L stream without its RIFF container. libwebp reads one, but a WebP file always has the container.

The tests read 88 files that assets/webp/make_webp.py wrote with libwebp's cwebp, some damaged on purpose. Each one must have the bytes that libwebp's dwebp gives, or be one of the files that libwebp refuses.

Color space of a decoded file

The file says	color_space
Nothing	SRGB
gAMA of 100000	LINEAR
gAMA near 45455	SRGB, as an approximation
An ICC profile, or another gamma	UNKNOWN_SPACE

texture_from refuses UNKNOWN_SPACE unless you pass color_space=SRGB or LINEAR. The gamma reading is approximate: a pure power differs from the sRGB curve by about a percent at midtones.

Integrity checks

The decoder checks each chunk's CRC, the zlib Adler-32, and that the output has exactly the size the header promised. It checks chunk order, duplication and length, and it refuses an unknown critical chunk. See Why the PNG reader checks structure.