Skip to content

v0.2.0

Choose a tag to compare

@misterekkk misterekkk released this 22 Oct 23:42

First release on PyPI:

pip install pylignum   # imported as `import lignum`

lignum predicts with LightGBM and XGBoost models, and its results are bit-identical to booster.predict(...).

Highlights

  • Exact semantics. Covers LightGBM and XGBoost thresholds, NaN and zero-as-missing, categorical splits, multiclass models and XGBoost's float32 arithmetic. Every test checks bit-exact parity against the real libraries.
  • Two engines. A binned interpreter handles any batch size. The compiled mode (model.compile()) turns the trees into native code with the system C compiler in a few seconds:
    • save the result as a library next to the model and load it in about 2 ms without a compiler;
    • model.autotune(X) picks the interpreter or the compiled code per batch size.
  • Formats. LightGBM model.txt and JSON, XGBoost JSON and .ubj, detected from the file content. model.save("model.bin") writes a binary file that loads by memory mapping in under a millisecond.
  • Arrow input. pyarrow tables and batches, polars data frames and any object with the Arrow PyCapsule interface are read without copies. Dictionary columns are treated as categories.
  • C API for other languages, including Arrow input and the compiled mode.
  • Wheels for Linux x86_64 and aarch64 (manylinux_2_28), macOS 15+ (arm64 and x86_64) and Windows x64. Python 3.10, 3.11, and abi3 for 3.12+.

Performance

LightGBM, 500 trees, Hetzner AMD EPYC-Milan (4 dedicated vCPU). Whole data set, all threads:

lignum compiled lleaves LightGBM
California housing 0.014 s 0.021 s 0.140 s
Covertype 0.265 s 0.269 s 2.401 s
Diabetes 130-US 0.151 s 0.803 s 1.830 s

On one thread, a single row takes 2.9-3.2 µs (lleaves 7.8-20 µs). The benchmarks page has full results for x86-64, ARM and Apple M4, including the cases where lignum is not the fastest.

Not supported

LightGBM linear trees and random forests, and multi-target / vector-leaf models. These models are rejected at load time.

Documentation: https://misterekkk.github.io/lignum/