Skip to content
This repository was archived by the owner on Apr 7, 2026. It is now read-only.

Entrenar v0.1.0 - Training & Optimization Library

Choose a tag to compare

View all tags
@noahgift noahgift released this 21 Nov 07:46
· 1165 commits to main since this release
v0.1.0
3bdfa45

Entrenar v0.1.0 - Production Release

Entrenar (Spanish: "to train") is a high-performance Rust library for training and optimizing neural networks with automatic differentiation, state-of-the-art optimizers, and memory-efficient LoRA/QLoRA fine-tuning.

🎉 First Production Release

This is the initial production-ready release of Entrenar, featuring a complete training infrastructure built with EXTREME TDD methodology.

✨ Features

Core Framework

  • Autograd Engine - Tape-based automatic differentiation with backward propagation
    • Tensor abstraction with gradient tracking
    • BackwardOp trait for custom operations
    • Attention, matmul, softmax, layer norm operations
    • Property-based gradient checking (200K+ iterations)

Optimizers

  • SGD with momentum support
  • Adam optimizer with bias correction
  • AdamW with decoupled weight decay
  • Gradient clipping via L2 norm
  • Learning rate scheduling (Cosine, Linear)
  • SIMD acceleration for parameter updates via Trueno
  • Convergence property tests for all optimizers

LoRA & QLoRA

  • LoRA layers with configurable rank and alpha
  • QLoRA with 4-bit quantized base weights
  • Adapter management (save/load separately from base model)
  • Memory benchmarks showing 4× reduction with QLoRA
  • Gradient flow tests ensuring proper backpropagation

Model Merging (Arcee Methods)

  • TIES (Task Inference via Elimination and Sign voting)
  • DARE (Drop And REscale with Bernoulli masking)
  • SLERP (Spherical Linear intERPolation)
  • Property tests for permutation invariance
  • Multi-model ensemble support

Knowledge Distillation

  • Temperature-scaled KL divergence loss
  • Multi-teacher ensemble distillation
  • Progressive layer-wise distillation
  • 44 distillation tests including 13 property tests
  • Temperature smoothing validation

Training Loop

  • High-level Trainer abstraction
  • Batch processing with configurable batch size
  • Metrics tracking (loss history, learning rates, steps)
  • Gradient clipping integration
  • Learning rate scheduling during training

Model I/O

  • Save/load models with multiple formats (JSON, YAML)
  • ModelMetadata with custom fields
  • Round-trip integrity validation
  • Automatic format detection from file extension

Declarative Configuration

  • YAML-based training configuration (Ludwig-style)
  • Schema validation with comprehensive error messages
  • Auto-inference of feature types from data
  • Single-command training via train_from_yaml()
  • Builder pattern for optimizers and models from config

📊 Quality Metrics

  • 258 tests passing (100% success rate)
    • 130 core library tests
    • 18 gradient checking tests
    • 35 architecture tests
    • 16 I/O and configuration tests
    • 13 property-based tests (13,000+ iterations)
    • 15 chaos engineering tests
    • 11 memory benchmark tests
    • 10+ additional integration tests
  • 0 clippy warnings (strict mode)
  • 0 TODOs remaining in codebase
  • 55 Rust source files with full documentation
  • 146 chapter mdBook documentation

🚀 Getting Started

Add to your Cargo.toml:

```toml
[dependencies]
entrenar = "0.1.0"
```

Quick example:

```rust
use entrenar::{Tensor, train::{Trainer, TrainConfig, MSELoss}, optim::Adam};

// Create trainer
let config = TrainConfig::new().with_log_interval(100);
let optimizer = Adam::default_params(0.001);
let mut trainer = Trainer::new(parameters, optimizer, config);
trainer.set_loss(Box::new(MSELoss));

// Train
let avg_loss = trainer.train_epoch(batches, |x| model.forward(x));
```

📚 Documentation

🔗 Links

🙏 Acknowledgments

Built with:

  • trueno 0.4.1 - SIMD-accelerated compute engine
  • ndarray 0.16 - N-dimensional arrays
  • serde 1.0 - Serialization framework

Part of the PAIML (Pragmatic AI Labs ML) ecosystem.

Quality Grade: A+ (Zero defects, 258 passing tests, EXTREME TDD methodology)

Assets 2
Loading