Repository navigation
Releases
NTransformer v1.4.0
NTransformer v1.4.0
First public release of NTransformer — Neuratal's independently developed, PyTorch-based, decoder-only Transformer architecture and training framework.
Not a pretrained model. No weights are distributed with this release; no quality claims are made.
This is a stable software release: tests green, docs complete, benchmark harness included, honest scope documentation.
What NTransformer is
A research/engineering core for decoder-only language models — architecture, inference, training engine, checkpointing, benchmarks — built around NTokenizer v0.3-polished (byte-level BPE, vocabulary 262,144, token IDs consumed verbatim). Full integration is fingerprint-verified at load, resume and inference time. The NTokenizer artifact is bound to the exact checkpoint, and mismatch is rejected explicitly. NTokenizer is a separate project (github.com/Neuratal/neuratal-tokenizer) and is not modified by this package.
Highlights
- Architecture: decoder-only, pre-norm; RMSNorm (fp32 stats; fused
F.rms_normfast path); RoPE (NeoX rotate-half), offset-aware KV-cache; Grouped-query attention (SDPA); SwiGLU; factorized tied embeddings (V×F input/output projection) - MoE (opt-in): independent SwiGLU experts, fp32 top-k softmax router with normalized gates, gather-once/scatter-once dispatch
- Training engine: deterministic resume (model + optimizer + scheduler + RNG + GradScaler + token counters); warmup → cosine; bf16/fp16 autocast with unscale-before-clip; MoE auxiliary losses; held-out validation split; full-resume determinism; 2-process CPU gloo DDP verified (dense and MoE); multi-GPU/NCCL not verified on dev hardware
- Long context (16K/32K): memory-bounded paths — gradient checkpointing + chunked linear cross-entropy (never materializes 262k-vocab logits) + last-position-only forward
- Checkpointing: safetensors + config.json, atomic writes everywhere, strict structural validation; local sharded safetensors for large models (not distributed checkpointing)
- Benchmarks: reproducible seed-42 harness; every number in this repo is measured on the dev machine (RTX 4060 Laptop); inter-run spread is normal on a 45W mobile GPU; nothing is estimated for other hardware
- Tests: 460 passed, 3 skipped (CUDA-dependent skips); synthetic data only for training tests; no corpus is bundled
Install
pip install neuratal-transformerPython ≥ 3.10. Runtime dependencies: torch>=2.4, safetensors>=0.4.
Documentation
- Architecture —
docs/architecture.md - Context length (16K/32K) —
docs/context-length.md - Inference —
docs/inference.md - Training —
docs/training.md - Tokenizer integration —
docs/tokenizer-integration.md - Comparison with HF/TE/Megatron-LM/nanoGPT —
docs/comparison.md - Benchmarking —
docs/benchmarking.md - Limitations / honest scope —
docs/limitations.md - Changelog —
CHANGELOG.md
License
Apache-2.0 — see LICENSE and NOTICE. NTokenizer is a separate project (CC-BY-SA-4.0).
Known limitations
- No trained weights. Development preset outputs gibberish.
- Production training is out of scope for this release; 32K paths are experimental.
- Multi-GPU/NCCL is not verified on dev hardware (no multi-GPU on this machine).
- No cross-hardware benchmarks exist; all numbers are from the dev machine.
- Corpus licensing is the caller's responsibility.
Honest scope
- This is not a GPT-4 replacement, and no product claims are made.
- This is not a Megatron-LM replacement — Neuratal recommends Megatron-LM for multi-node scale training, Hugging Face Transformers for production serving, nanoGPT for teaching.