NSZ compatibility

The app writes and reads the NSZ and XCZ format of nsz, the one DBI and Tinfoil install. It is written in Go for this project from the format itself, so nothing else needs to be installed. This page explains how it works, how it was tested on a console, and how it deals with the problems known in nsz.

The format in short

An NSZ is an NSP whose large NCA files are replaced by NCZ files. An NCZ keeps the first 0x4000 bytes of the NCA as they are (its header, still encrypted, with Nintendo's signature), then a table with the offset, size, key and AES counter of each section, then the rest of the NCA decrypted and compressed with zstd. To install it, the installer decompresses the data, encrypts each section again with its key and counter, and gets the original NCA back byte for byte: its SHA-256 has to match the content ID in its name.

NSZ files are one zstd stream per NCA. XCZ files (and NSZ files written in blocks) are split in blocks of 1 MB that can be read on their own. The app keeps the zstd window at 8 MB, so a console never runs short of memory while installing.

What the app checks

1. Before compressing, every NCA of the original is hashed.
2. Each large NCA is decrypted with its title key, from its ticket or from your title.keys, and compressed.
3. The new file is decompressed again, every section encrypted again, and every NCA, ticket and certificate has to give back the original SHA-256. None may be missing.
4. Only then does the file get its name (it is written under a hidden temporary name until then). The original is deleted only if you asked for it and the check went through every file.

Merged packs ("SuperNSP") need one more step. Tools such as NSC_Builder rewrite the header of some NCA files when they build them, so their content ID no longer matches, and the hash in the metadata (CNMT) is the old one. The app does not use that stale hash: it checks the hash trees stored in the NCA header, the way hactool does. When every hash matches, the file is marked as rewritten but fine; otherwise it is damaged.

Tested on a console

A game with its update and DLC was compressed by the app at the Maximum level and installed with DBI over FTP on firmware 22.1.0:

• Base game, about 7 GB compressed: TRANSFER OK, SIGNATURE OK, the game boots.
• Update, an NCZ with 866 sections: TRANSFER OK, SIGNATURE OK, the console shows the version of the update. This is the hardest case for the format: every section is encrypted again with its own counter, and one wrong section is enough to break the hash.
• DLC, about 3 GB compressed: TRANSFER OK, SIGNATURE OK, listed as installed.
• The same three as one complete pack (a single NSZ): installed, every content registered, the game boots with its update.

TRANSFER OK is DBI rebuilding each NCA from the NCZ and finding the hash of its content ID. SIGNATURE OK is the header signature, which the format never touches.

Known problems of nsz

These are the issues reported in nsz that touch compression or decompression, and how the app handles each one. Each case has a test in the code.

nsz issue	What happened	In this app
#210	A size that is an exact multiple of the block size gave a last block of 0 bytes and broken data	The last block is the smaller of the block size and what is left, so it is a full block
#79, #190	A short last block stored without compression was read wrongly	Exactly the bytes of the block table are read, and "compressed or not" is told by the real size of the block
#247, #255	A first section starting after, or inside, the first 0x4000 bytes gave wrong sizes	The gap is filled as nsz writes it, and the sizes of the NSP come from what was really written
#99, #189, #193	A game without a ticket crashed (can't concat NoneType to bytes)	A clear message that the title key is missing, and the game is listed in Issues
#193	Merged packs reported as CORRUPTED because of the stale CNMT hash	The hash trees of the NCA are checked instead (see above)
#124, #110	The original could be deleted, or "Done" shown, after a failure	The task fails and the original stays; it is deleted only after a full check
#254	Levels 21 and 22 wrote nothing	Only three named levels, no invalid value possible
#100	Games with sparse storage were not detected	They are told apart and left as they are; they install normally

Deliberate differences

• The zstd window stays at 8 MB. NSZ files made with a large window (128 MB is common) decompress fine in the app, but a console has less memory to spare.
• Maximum is a bit lighter than the default of nsz. nsz can make files a few % smaller with much more time and memory. Installers read every level the same.
• The delta fragments of updates are kept. An NSZ decompresses to the same NSP it came from; complete packs leave them out, since installs from a file never use them.
• The CNMT hash is not used to check a file. It is the one merging tools leave stale; the round trip and the hash trees check the content instead.