Skip to content

Developer Guide

Jump to bottom
Nanook edited this page Sep 24, 2026 · 1 revision

For developers who want to embed NKit or NKDS in their own projects, understand the codebase, or contribute.

Project Status

The source code is available on GitHub. Contributions and bug reports are welcome.

Note: This documentation is a first draft and will be improved over time. If something is unclear or missing, please open an issue or ask on Discord.

Libraries at a Glance

Library Assembly Namespace Purpose
NKit NKitLib Nanook.NKit Disc image processing engine — convert, extract, scan, verify, fix, dedupe
NKitDataStore NKitDataStore NKitDataStore Block storage engine — content-addressed shards, index, compaction
NKDS NKDSLib NKDS DataStore product — set management, VFS mount orchestration
NKitCore (dependency) NKitCore Generic async section pipeline (StreamBlockCore)
NKitStream (dependency) NKitStream BufferStream and streaming primitives

Planned NuGet packages: NKit · NKitDataStore · NKDS — not yet published. See the Roadmap for details.

NuGet packages are planned for a future release. See the Roadmap for details.

Target Frameworks

Library Frameworks
NKit, NKitDataStore, NKitCore, NKitStream net10.0, net8.0, netstandard2.1
NKDS, NKDS.UI, NKit.UI net10.0
CLI apps (nkit, nkds) net10.0 (AOT-published)

Key Design Constraints

AOT-safe code only

All libraries are designed for .NET Native AOT. This means:

  • No reflection in any read/write/parse/process path
  • No runtime code generation (no Emit, no expression trees in hot paths)
  • No reflection-based serialisers (no System.Text.Json or YamlDotNet in format I/O paths) — all parsing is hand-written
  • Custom YAML reading/writing uses AotYamlSerializer in NKit.UI/Helpers/Yaml/

If you extend NKit with a new format reader, step, or container decoder, it must follow these constraints.

Explicit endianness

All disc image formats and NKDS's own binary format use big-endian byte order on disk. Use the explicit endian helpers from NKit/Common/Utils.cs:

// Reading
uint value = buffer.ReadUInt32B(offset);   // big-endian
uint value = buffer.ReadUInt32L(offset);   // little-endian

// Writing
buffer.WriteUInt32B(offset, value);

Never use BitConverter.ToUInt32(bytes, offset) directly — it is host-endian and will produce wrong results on any non-x64 host, and inconsistent results across platforms.

Forward-reading streams

Source images are read as forward-only streams. Decoders and format readers must not seek backwards in the raw source. If you need to re-read a region, the BufferStream cache handles it transparently — but only for regions within the current cache window. Design format readers to be forward-sequential.

Processing a Disc Image with NKit

The simplest entry point is NKitProcessor. It accepts an AppSettings (loaded from a YAML config or built programmatically) and a source file path:

// Load settings from a config file
AppSettings settings = new AppSettings("path/to/nkit.yaml", overrides: null);

// Process a file
NKitProcessor processor = new NKitProcessor(settings, sourceFilePath, logSink);
NKitTaskResults results = processor.Process(cancellationToken);

// Check results
foreach (NKitTaskResult result in results.Results)
{
    Console.WriteLine($"{result.InFile}: {result.VerifyResult}");
}

AppSettings can also be constructed programmatically by passing a dictionary of override key-value pairs as the second argument, without a config file:

Dictionary<string, string> overrides = new()
{
    ["task"]    = "convert",
    ["convert"] = "rvz:zstd:19:128k:16",
    ["out"]     = "/output/path",
};
AppSettings settings = new AppSettings(configPath: null, overrides);

Working with NKDS DataStore Directly

// Open (or create) a set
using DataStore store = DataStore.Open("/data/nkit", setName: "wii");

// List images
IReadOnlyList<ImageRecord> images = store.ListImages();

// Export an image
OperationResult result = store.ExportImage(imageId, outputPath, format: "rvz:zstd:19");

The DataStore class in NKitDataStore is the main entry point for storage operations. It is thread-safe for concurrent reads and enforces single-writer per set.

Adding a New Format (Container Decoder)

New container decoders implement IAsIso — the interface that presents any source as a raw ISO stream to the readers above:

public interface IAsIso : IDisposable
{
    Stream Stream { get; }   // the ISO-like stream
    long Size { get; }       // logical size
    SystemType SystemType { get; }
    // ...
}

Implement IAsIso.Create(byte[] header) as a static factory that returns null if the header bytes don't match your format. Register the factory in ContainerFactory alongside the existing decoders.

The container decoder wraps the source stream and translates reads. It must be forward-reading (no seeks into source), endian-explicit, and AOT-safe.

Adding a New Processing Step

Steps implement INKitStep and are instantiated per task:

public interface INKitStep
{
    string Name { get; }
    bool CanProcess(SystemType system, TaskType task);
    void Process(NKitStepContext context, ISection section);
    void Complete(NKitStepContext context);
}

Steps receive sections in output order (serial, ordered) on the Output stage thread. They must not assume anything about threading within Process — multiple sections may be processing concurrently in the stages above, but by the time a section reaches INKitStep.Process it is always serial.

Interactive CLI Prompting

Both CLI apps use a shared IPrompter abstraction from NKit/Common/Interactive/IPrompter.cs. This lets interactive builders be unit-tested with a scripted prompter:

public class MyInteractiveFlow
{
    private readonly IPrompter _p;

    public MyInteractiveFlow(IPrompter prompter) => _p = prompter;

    public string[] Build()
    {
        string choice = _p.Select("Pick a task", new[] { "convert", "scan", "verify" });
        // ...
    }
}

// In tests:
var prompter = new ScriptedPrompter(new[] { "convert", "yes" });
var flow = new MyInteractiveFlow(prompter);
string[] result = flow.Build();

The Spectre.Console-backed implementation (Shared/Cli/SpectrePrompter.cs) is compiled into each CLI app via a linked source file, keeping the core libraries free of any Spectre dependency.

Testing

Tests use xUnit with FsCheck for property-based tests. The test projects are:

Project Scope
NKit.Tests CLI parser, configuration, logging, settings
NKitDataStore.Tests DataStore operations, compaction, crash recovery, VFS
NKDS.Tests NKDS operations, dat verification
NKitCore.Tests Pipeline section engine
NKitStream.Tests BufferStream, file scanning

Wiped image tests (NKit.Tests/WipedImageTests/) are integration tests that run the full pipeline against reference disc images. They require the test fixture files and are not run in CI by default — they take ~40 minutes.

Run a filtered subset:

dotnet test NKit.Tests --filter "FullyQualifiedName~CliParserTests" --no-restore

dotnet test NKit.Tests --filter "Trait=Area~Configuration" --no-restore

Code Style

  • C# latest language version
  • Nullable reference types enabled in newer projects
  • No #region blocks
  • Prefer explicit types over var in non-obvious cases
  • All binary I/O through explicit endian helpers (see above)
  • No reflection in format/processing hot paths

Further Reading

  • Architecture — Design principles and how the pieces fit together
  • NKDS Storage Model — Block store design and crash safety
  • NKit CLI — Full parameter reference (same parameters available programmatically)
  • Roadmap — Planned features including NuGet library releases

Clone this wiki locally