Skip to content

v0.1.0

Choose a tag to compare

View all tags
@nisbenz nisbenz released this 27 Jul 12:29
· 207 commits to main since this release
v0.1.0
954d8fd

Changelog

[0.1.0] — 2026-07-27

Initial release of TensorLib — a from-scratch deep learning tensor library written entirely in C with zero external ML dependencies.

Added

  • Tensor Core — N-dimensional float32 arrays with NumPy-style broadcasting, zero-copy strided views, reshape, transpose, slice, squeeze, expand, and reference-counted storage with mutation version tracking.
  • Tensor Operations — Element-wise arithmetic (add, sub, mul, div, pow, neg, exp, log, sqrt), activations (ReLU, sigmoid, tanh, GELU), reductions (sum, mean, max), gather, and generalized strided matmul.
  • AVX2+FMA Matmul — Blocked micro-kernel (MR=4, NR=16) with packed-RHS optimization and batched strided dispatch. Tile sizes MC=64, NC=64, KC=128. Benchmarked against OpenBLAS.
  • Autograd Engine — Dynamic reverse-mode automatic differentiation over 23 differentiable operations. Stale-graph detection via storage version counters. Transactional gradient accumulation with topological traversal.
  • Neural Network Modules — Composable C-style OOP module hierarchy with function-pointer dispatch:
    • Linear (with bias), Embedding, learned Positional Embedding, LayerNorm, Dropout
    • Multi-Head Causal Self-Attention (fused QKV projection, causal masking)
    • Pre-Norm Decoder Block, full GPT-style Decoder stack
  • Loss Functions — Cross-entropy (numerically stable), Softmax, LogSoftmax.
  • Optimizers — SGD with momentum, AdamW with decoupled weight decay, bias correction, and gradient clipping.
  • Checkpointing — Versioned, atomic (transaction-safe) binary save/load of model parameters, optimizer state, and RNG state.
  • Deterministic RNG — Splitmix64 PRNG with uniform and normal (Box-Muller) distributions.
  • Examples:
    • TinyLM — Byte-level decoder language model (~1.9M params, 4-layer, 192-width, 6-head, 128-context) trained with AdamW on raw byte corpora. Includes checkpoint/resume and text generation.
    • MNIST MLP — 784→128 ReLU→10 Softmax classifier trained with SGD and cross-entropy.
    • Autograd Example — Computation graph visualization demo.
  • Test Suite — 33 unit test executables covering tensor ops, autograd, all NN modules, optimizers, and checkpointing.
  • CI — GitHub Actions workflow running full test suite on Alpine Linux.

Build System

  • CMake and GNU Makefile with -O3 -march=native -flto -fopenmp optimizations.
  • OpenMP-accelerated matmul and element-wise kernels.
Assets 3
Loading