Skip to content

v0.3.0: SafeTensors Model Serialization

Choose a tag to compare

View all tags
@noahgift noahgift released this 19 Nov 08:35
· 4199 commits to main since this release
v0.3.0
9604f1e

Release v0.3.0: SafeTensors Model Serialization

Release Date: 2025-11-19

This release adds industry-standard SafeTensors serialization support, enabling production deployment of aprender models to inference engines like realizar, Ollama, and integration with HuggingFace, PyTorch, and TensorFlow ecosystems.

🎯 Major Features

SafeTensors Serialization

LinearRegression (Issue #5)

  • ✅ LinearRegression::save_safetensors() - Export models to SafeTensors format
  • ✅ LinearRegression::load_safetensors() - Load models from SafeTensors format
  • ✅ 7 comprehensive tests (roundtrip, validation, error handling)

LogisticRegression (Issue #6)

  • ✅ LogisticRegression::save_safetensors() - Export binary classification models
  • ✅ LogisticRegression::load_safetensors() - Load models from SafeTensors format
  • ✅ 5 comprehensive tests (unfitted, roundtrip, corrupted, missing, probability preservation)

🚀 What is SafeTensors?

SafeTensors is the industry-standard format for ML model serialization:

  • Zero-copy loading - Efficient memory usage
  • Cross-platform - Compatible with Python, Rust, JavaScript
  • Language-agnostic - Works with all major ML frameworks
  • Safe - No arbitrary code execution (unlike pickle)
  • Deterministic - Reproducible builds with sorted keys

Binary Format:

┌─────────────────────────────────────────────────┐
│ 8-byte header (u64 little-endian)              │
├─────────────────────────────────────────────────┤
│ JSON metadata: tensor names, dtypes, shapes    │
├─────────────────────────────────────────────────┤
│ Raw F32 tensor data (IEEE 754 little-endian)   │
└─────────────────────────────────────────────────┘

💡 Usage Examples

LinearRegression

use aprender::linear_model::LinearRegression;
use aprender::prelude::*;

// Train model
let mut model = LinearRegression::new();
model.fit(&x_train, &y_train).unwrap();

// Save to SafeTensors
model.save_safetensors("regression_model.safetensors").unwrap();

// Load and deploy
let loaded = LinearRegression::load_safetensors("regression_model.safetensors").unwrap();
let predictions = loaded.predict(&x_test);

LogisticRegression

use aprender::classification::LogisticRegression;
use aprender::prelude::*;

// Train binary classifier
let mut model = LogisticRegression::new()
    .with_learning_rate(0.1)
    .with_max_iter(1000);
model.fit(&x_train, &y_train).unwrap();

// Save to SafeTensors
model.save_safetensors("classifier.safetensors").unwrap();

// Load and get probabilities
let loaded = LogisticRegression::load_safetensors("classifier.safetensors").unwrap();
let probabilities = loaded.predict_proba(&x_test);  // Exact preservation!

🌐 Production Deployment

Deploy to realizar Inference Engine

# 1. Train model in aprender
cargo run --example train_model

# 2. Model saved as model.safetensors

# 3. Deploy to realizar
realizar upload model.safetensors \
    --name "production-model-v1" \
    --version "1.0.0"

# 4. Inference via REST API
curl -X POST http://realizar:8080/predict/production-model-v1 \
    -d '{"features": [1.5, 2.3, 3.7]}'

Integration with HuggingFace

from safetensors import safe_open

# Load aprender model in Python
tensors = {}
with safe_open("model.safetensors", framework="pt") as f:
    for key in f.keys():
        tensors[key] = f.get_tensor(key)

print(tensors["coefficients"])  # torch.Tensor([...])
print(tensors["intercept"])     # torch.Tensor([...])

📊 Quality Metrics

Testing

  • Total Tests: 417 unit tests + 63 doctests (all passing)
  • New Tests: 12 SafeTensors tests (7 LinearRegression + 5 LogisticRegression)
  • Coverage: 100% for serialization methods
  • Mutation Testing: All mutants caught

Code Quality

  • ✅ Zero clippy warnings
  • ✅ Zero rustdoc warnings
  • ✅ All PMAT pre-commit hooks passing
  • ✅ Complexity: All functions ≤10 cyclomatic complexity
  • ✅ SATD: Zero TODO/FIXME/HACK comments

Documentation

  • model-serialization.md: 685-line comprehensive chapter
  • logistic-regression.md: Extended with 281-line SafeTensors section
  • RED-GREEN-REFACTOR case studies
  • Production deployment examples

🔧 Technical Details

Key Design Decisions

1. Deterministic Serialization (BTreeMap)

  • Uses BTreeMap instead of HashMap for sorted keys
  • Ensures byte-for-byte reproducible builds
  • Git diffs show only real changes

2. Probability Preservation (LogisticRegression)

  • Binary classification requires exact IEEE 754 F32 equality
  • Critical for medical diagnosis, financial fraud detection
  • Test verifies probabilities identical after save/load roundtrip

3. Hyperparameters Not Serialized

  • Training hyperparameters (learning_rate, max_iter) only affect training
  • Only weights (coefficients + intercept) serialized
  • Smaller file size, compatible with frameworks that don't support hyperparameters

📦 Installation

Add to your Cargo.toml:

[dependencies]
aprender = "0.3.0"

Or install via cargo:

cargo add aprender@0.3.0

🔗 Ecosystem Compatibility

✅ HuggingFace - Load aprender models in Python/PyTorch
✅ Ollama - Convert to GGUF for local LLM integration
✅ PyTorch/TensorFlow - Cross-framework model sharing
✅ realizar - Rust-native inference engine
✅ GGUF/ONNX - Conversion-ready format

📝 Changes Since v0.2.0

Added

  • LinearRegression::save_safetensors() / load_safetensors() (Issue #5)
  • LogisticRegression::save_safetensors() / load_safetensors() (Issue #6)
  • SafeTensors module (src/serialization/safetensors.rs)
  • 12 new tests for SafeTensors serialization
  • Extended book chapters with 966 lines of documentation

Changed

  • Dependencies: Added serde_json = "1.0" for SafeTensors metadata
  • Test count: 417 lib tests (up from 412)

Fixed

  • None

🎓 Learning Resources

Book Chapters

Documentation

🙏 Acknowledgments

This release implements the SafeTensors format specification by HuggingFace, enabling cross-language ML model deployment.

Built with EXTREME TDD methodology: RED-GREEN-REFACTOR.

📋 Full Changelog

View the complete CHANGELOG for detailed changes.

Minimum Rust Version: 1.70
Breaking Changes: None
Migration Guide: Not required (backward compatible)

Assets 2
Loading