Skip to content

NTransformer v1.4.0

Latest

Choose a tag to compare

@Neuratal Neuratal released this 19 Nov 11:44
· 2 commits to master since this release

NTransformer v1.4.0

First public release of NTransformer — Neuratal's independently developed, PyTorch-based, decoder-only Transformer architecture and training framework.

Not a pretrained model. No weights are distributed with this release; no quality claims are made.
This is a stable software release: tests green, docs complete, benchmark harness included, honest scope documentation.

What NTransformer is

A research/engineering core for decoder-only language models — architecture, inference, training engine, checkpointing, benchmarks — built around NTokenizer v0.3-polished (byte-level BPE, vocabulary 262,144, token IDs consumed verbatim). Full integration is fingerprint-verified at load, resume and inference time. The NTokenizer artifact is bound to the exact checkpoint, and mismatch is rejected explicitly. NTokenizer is a separate project (github.com/Neuratal/neuratal-tokenizer) and is not modified by this package.

Highlights

  • Architecture: decoder-only, pre-norm; RMSNorm (fp32 stats; fused F.rms_norm fast path); RoPE (NeoX rotate-half), offset-aware KV-cache; Grouped-query attention (SDPA); SwiGLU; factorized tied embeddings (V×F input/output projection)
  • MoE (opt-in): independent SwiGLU experts, fp32 top-k softmax router with normalized gates, gather-once/scatter-once dispatch
  • Training engine: deterministic resume (model + optimizer + scheduler + RNG + GradScaler + token counters); warmup → cosine; bf16/fp16 autocast with unscale-before-clip; MoE auxiliary losses; held-out validation split; full-resume determinism; 2-process CPU gloo DDP verified (dense and MoE); multi-GPU/NCCL not verified on dev hardware
  • Long context (16K/32K): memory-bounded paths — gradient checkpointing + chunked linear cross-entropy (never materializes 262k-vocab logits) + last-position-only forward
  • Checkpointing: safetensors + config.json, atomic writes everywhere, strict structural validation; local sharded safetensors for large models (not distributed checkpointing)
  • Benchmarks: reproducible seed-42 harness; every number in this repo is measured on the dev machine (RTX 4060 Laptop); inter-run spread is normal on a 45W mobile GPU; nothing is estimated for other hardware
  • Tests: 460 passed, 3 skipped (CUDA-dependent skips); synthetic data only for training tests; no corpus is bundled

Install

pip install neuratal-transformer

Python ≥ 3.10. Runtime dependencies: torch>=2.4, safetensors>=0.4.

Documentation

  • Architecture — docs/architecture.md
  • Context length (16K/32K) — docs/context-length.md
  • Inference — docs/inference.md
  • Training — docs/training.md
  • Tokenizer integration — docs/tokenizer-integration.md
  • Comparison with HF/TE/Megatron-LM/nanoGPT — docs/comparison.md
  • Benchmarking — docs/benchmarking.md
  • Limitations / honest scope — docs/limitations.md
  • Changelog — CHANGELOG.md

License

Apache-2.0 — see LICENSE and NOTICE. NTokenizer is a separate project (CC-BY-SA-4.0).

Known limitations

  • No trained weights. Development preset outputs gibberish.
  • Production training is out of scope for this release; 32K paths are experimental.
  • Multi-GPU/NCCL is not verified on dev hardware (no multi-GPU on this machine).
  • No cross-hardware benchmarks exist; all numbers are from the dev machine.
  • Corpus licensing is the caller's responsibility.

Honest scope

  • This is not a GPT-4 replacement, and no product claims are made.
  • This is not a Megatron-LM replacement — Neuratal recommends Megatron-LM for multi-node scale training, Hugging Face Transformers for production serving, nanoGPT for teaching.