Writing A Kernel

Writing a Kernel

This repo is a factory: 26 packages built from one template. This page is the bar a new kernel must clear and the shape it must take. Also read CONTRIBUTING.md and How It Works.

1. Pick a target — the evidence bar

A candidate library qualifies when all of these hold:

• A pure-Python or pure-JavaScript hot loop. The reference spends its time in interpreter-executed arithmetic, not in already-native code (NumPy/BLAS calls, C extensions) or I/O. Prove it with a profile, not a hunch: e.g. Fuse.js showed ~94% of query CPU in the Bitap DP; simple-statistics showed 69.8% in the DP cell fill. If the hot loop is already native, there is nothing for a kernel to win.
• A popular, stable API worth matching. Drop-in means users change one import line. The reference package is the oracle; its releases must be pinnable.
• A well-defined published algorithm. The kernel is written from the algorithm's description (textbook formulas, RFCs, papers), never from the reference's source.
• An integration mode, chosen up front: drop-in replacement (same API — bm25, fuse, jsonschema) or alongside (mirrored functions, no monkeypatching — cclib's cclib_integration, MetPy's grid entry point). Both exist on main; pick the one the ecosystem can actually adopt.

Rule of thumb from the autopsies: a pure-interpreter loop is typically a 10–100× opportunity; an interpreter loop over large data with a batchable shape is 100×+. Publish your profile in the proposal issue.

2. The clean-room rule (non-negotiable)

Kernels implement the published algorithm from its description — not from the reference package's source code. The reference package is used only as a test and benchmark oracle (pinned in pixi.toml), never as a runtime dependency. The one sanctioned exception is vendoring the reference as the fallback backend under its own license with a NOTICE (Fuse.js, simple-statistics, natural, MiniSearch) — that is redistribution with attribution, not derivation. Contributions are inbound = outbound under Apache-2.0; no CLA, no DCO.

3. The batch C-ABI shape

<name>mojo_abi_version()      # handshake — checked before any native call
<name>_create(...)            # index/context build, once
<name>_score(...)             # the whole workload per call — never per item
<name>_destroy(...)

FFI cost must be per call, not per item: one call scores the whole corpus, evaluates the whole grid, smooths the whole spectrum. A kernel that crosses the boundary per element loses to serialization no matter how fast it is — that is precisely the jmespath story, and we kept the receipt. Changing a signature means bumping the ABI version and teaching the loader to refuse mismatched libraries. Follow the factory layout: kernels/<name>/src/*.mojo + build.sh (mojo build --emit shared-lib), wrapper with _native.py loader + _reference.py vendored fallback (Python) or the koffi + optionalDependencies workspace (TypeScript).

4. The differential-test requirement

No kernel lands without a differential suite that:

• asserts the native backend and the forced fallback (<NAME>_MOJO_DISABLE_NATIVE=1) against the real reference package, element-wise, at a documented per-kernel tolerance (existing precedents: bm25 1e-8 absolute; gaussgrid 1e-10 relative; fuse scores within 1e-9 with identical match spans; several kernels bit-exact);
• covers the reference's edge semantics, not just the happy path — NaN/warmup prefixes, exact exception classes, -0.0, int-vs-float, unicode, empty inputs, ordering and tie-breaking;
• runs on ubuntu-latest and macos-latest in CI, both passes. Widening a tolerance requires justification in the PR.

5. The benchmark requirement

• Seeded, synthetic, reproducible corpora (no redistributed datasets), in benchmarks/bench_<name>.*.
• Correctness gate before every timing pass. Median of 5 runs. Cold and warm reported separately — cold is the first call in a fresh process.
• A machine/environment block (CPU, OS, Python/Node, oracle version, Mojo version, date).
• Honest ranges: report the cells where the kernel loses, and compare against the fastest credible incumbent, not only the pure-language oracle (the "beat the natives" rule — see Roadmap).

6. The fallback requirement

Every package vendors a pure-language reference implementation, selected automatically on any load/ABI failure and forceable via env var. It must pass the same differential suite at the same tolerance — the fallback is the Windows product, not a stub. Wheels/platform packages are per-platform and self-contained (delocate / auditwheel repair / patchelf vendor the Mojo runtime); there is no sdist.

7. Landing it

Branch feat/<name> from main, Conventional Commits with the kernel as scope (feat(<name>): …), one focused diff. CI must be green — every workflow is a thin wrapper over a pixi run task, so run it locally first. Review follows CODEOWNERS; pixi.lock, packaging scripts, and .github/ always get deliberate maintainer review. Add the package to the kernel table in the repo README and regenerate numbers with the bench scripts — never hand-edit.

Attribution

Everything here is Apache-2.0, © 2026 Algenta, built by the Algenta team. By submitting a PR you license your contribution under the project's existing license (inbound = outbound, per GitHub ToS §D.6). The clean-room rule is a legal requirement, not a style preference: only contribute work you have the right to submit.

---

Questions first? FAQ · GitHub Issues · How It Works