Repository navigation
v0.2.0
First release on PyPI:
pip install pylignum # imported as `import lignum`lignum predicts with LightGBM and XGBoost models, and its results are bit-identical to booster.predict(...).
Highlights
- Exact semantics. Covers LightGBM and XGBoost thresholds, NaN and zero-as-missing, categorical splits, multiclass models and XGBoost's float32 arithmetic. Every test checks bit-exact parity against the real libraries.
- Two engines. A binned interpreter handles any batch size. The compiled mode (
model.compile()) turns the trees into native code with the system C compiler in a few seconds:- save the result as a library next to the model and load it in about 2 ms without a compiler;
model.autotune(X)picks the interpreter or the compiled code per batch size.
- Formats. LightGBM
model.txtand JSON, XGBoost JSON and.ubj, detected from the file content.model.save("model.bin")writes a binary file that loads by memory mapping in under a millisecond. - Arrow input. pyarrow tables and batches, polars data frames and any object with the Arrow PyCapsule interface are read without copies. Dictionary columns are treated as categories.
- C API for other languages, including Arrow input and the compiled mode.
- Wheels for Linux x86_64 and aarch64 (manylinux_2_28), macOS 15+ (arm64 and x86_64) and Windows x64. Python 3.10, 3.11, and abi3 for 3.12+.
Performance
LightGBM, 500 trees, Hetzner AMD EPYC-Milan (4 dedicated vCPU). Whole data set, all threads:
| lignum compiled | lleaves | LightGBM | |
|---|---|---|---|
| California housing | 0.014 s | 0.021 s | 0.140 s |
| Covertype | 0.265 s | 0.269 s | 2.401 s |
| Diabetes 130-US | 0.151 s | 0.803 s | 1.830 s |
On one thread, a single row takes 2.9-3.2 µs (lleaves 7.8-20 µs). The benchmarks page has full results for x86-64, ARM and Apple M4, including the cases where lignum is not the fastest.
Not supported
LightGBM linear trees and random forests, and multi-target / vector-leaf models. These models are rejected at load time.
Documentation: https://misterekkk.github.io/lignum/