Skip to content

NeuralOps v1.4.3: Multi-Dataset Streaming, Varlen Document Packing & Production CLI Suite

Choose a tag to compare

View all tags
@tahamajs tahamajs released this 19 Oct 07:27
· 7749 commits to main since this release
v1.4.3
7e6beef

🚀 NeuralOps v1.4.3: Multi-Dataset Streaming, Varlen Document Packing, WSD Scheduler & Production CLI Suite

Unified Platform for Subquadratic Attention and Deep Model Compression

131 canonical attention mechanisms · 72 compression engines · 114 native CUDA C++ kernels · 62 Triton hardware kernels · 27 master Jupyter notebooks

PyPI version
PyPI Downloads
License: MIT
Contributors
Python 3.10+
PyTorch 2.11+
CUDA 12.8+

🌟 Highlights

  • Multi-Dataset Streaming Ingestion: MultiDatasetLoader and parse_dataset_spec natively parse comma-separated corpora (e.g. --dataset HuggingFaceFW/fineweb-edu,opencsg/Fineweb-Edu-Chinese-V2), dynamically interleaving streaming and map-style iterators with weighted round-robin scheduling and offline synthetic fallback.
  • Variable-Length Document Packing: PackedDataset and pack_sequences concatenate arbitrary-length documents into fixed-budget sequences (e.g. 65,536 or 8,192 tokens) with zero padding overhead, returning cumulative document boundary vectors (cu_seqlens) required by Flash and linear attention kernels.
  • Warmup-Stable-Decay (WSD) Learning Rate Scheduler: Native WSDScheduler registered in OPTIMIZER_REGISTRY, providing linear warmup, prolonged stable peak learning rates, and smooth cosine/linear decay phases controlled by --lr_scheduler.decay_ratio.
  • Production-Grade CLI Suite: Comprehensive command handlers for train, eval, compress, serve, test, list, info, and doctor, featuring native Lion and AdamW optimizers, Rich terminal status displays, and structured JSON telemetry export.
  • Visual Overhaul & Navigation: Navigation pills, "NeuralOps At A Glance" executive matrix card, 30-second quickstart, and comprehensive CLI quick reference guide.
  • Repository Hygiene & Non-Core Untracking: Complete git isolation for non-core analysis and build folders (audit/, docs/, results/, academic/, paper/, thesis_final/, logs/, scratch/), purging tracked indices while preserving local research assets.
  • Strict Invariant Verification: All 131 attention mechanisms, 72 compression engines, 26 Triton kernel modules, 114 native CUDA C++ packages, and 27 executed master Jupyter notebooks verified 100% operational.

🏗️ Architectural Breakdown

1. Multi-Dataset Streaming & Document Packing

  • neuralops.data.multidataset: Support for comma-separated dataset specs, per-dataset configurations, streaming mode, and fallback synthetic generation.
  • neuralops.data.packing: Concatenates documents without waste, tracking exact cu_seqlens boundaries [0, len_1, len_1+len_2, ..., seq_len].
  • Integration into train_cmd.py for effortless training runs: 
    neuralops train \
  --model gpt2 \
  --dataset "HuggingFaceFW/fineweb-edu,opencsg/Fineweb-Edu-Chinese-V2" \
  --varlen \
  --seq-len 65536 \
  --context-len 4096 \
  --optimizer lion \
  --lr-scheduler wsd \
  --decay-ratio 0.1 \
  --steps 20480

2. Warmup-Stable-Decay (WSD) Optimizer Support

  • Stable learning rates during bulk pre-training preventing premature convergence.
  • Cosine or linear decay over the final decay_ratio fraction of total training steps down to lr_min.

3. Unified Production CLI Suite

  • neuralops train: Configurable training and fine-tuning engine with multi-dataset support, gradient accumulation, and optimizer options.
  • neuralops eval: Perplexity, latency, throughput, and VRAM benchmarking with Rich tables and --output-json.
  • neuralops compress: One-line model compression across SVD, RPCA, Tensor-Train, and INT4 quantization.
  • neuralops serve: Production serving engine utilizing PagedKVCache and speculative decoding.
  • neuralops list: Discover registered attentions and compressors filtered by family and backend.
  • neuralops doctor: Comprehensive hardware, compiler, and environment diagnostic checks.

📊 Benchmarks & Key Metrics (NVIDIA GeForce RTX 4070 SM 8.9)

Kernel / Subsystem Baseline (PyTorch Eager) NeuralOps Accelerated Speedup / Metric
hgrn2_scan 13.11 ms 0.083 ms 157.9x speedup
gla_scan 14.86 ms 0.104 ms 142.82x speedup
gdn_scan 12.39 ms 0.090 ms 137.69x speedup
PagedKVCache Serving Standard Contiguous KV Paged Non-Contiguous 77.5% memory savings
SelfSpeculativeDecoder Eager Autoregressive Speculative Draft Loop 7.59x speedup (1,422 tok/s)
Variable-Length Packing Padded Batches (70% padding) Zero-Padding Packed 3.3x effective token throughput

📦 Installation & Verification

# 1. Install NeuralOps v1.4.3 from PyPI
pip install neuralops==1.4.3

# 2. Or install full acceleration bundle
pip install "neuralops[cuda,export,eval,mlops]==1.4.3"

# 3. Verify installation and hardware diagnostics
neuralops doctor

# 4. Run built-in test suite
neuralops test --all

📝 Contributors & Community

We extend our sincere thanks to all contributors whose engineering, testing, and feedback made the v1.4.3 release possible:
👉 https://github.com/tahamajs/NeuralOps/graphs/contributors

Assets 2
Loading