learn is differentiable programming as ordinary Arturo data. Version 0.32 adds backend-native scatter assignment for mutable views and zero-copy reshape/transpose views.
import "learn"!
do [
xTrain: tensor [0.0 1.0 2.0 3.0]
yTrain: tensor [1.0 3.0 5.0 7.0]
model: graph [
x: input
y: input
w: parameter 0.0
b: parameter 0.0
linear: x * w
prediction: linear + b
residual: prediction - y
squared: square residual
loss: mean squared
]
optimizer: sgd.rate: 0.05 (parameters model)
batches: batcher.size: 2.shuffle: false xTrain yTrain
history: train.epochs: 250 model optimizer batches
print model\w
print model\b
print last history\epochs
]
The result converges to w ≈ 2 and b ≈ 1. Named intermediate expressions are deliberate: Arturo has no operator precedence, and explicit graph steps keep both evaluation and graph inspection unambiguous.
tensor value,tensor.zeros shape,tensor.ones shape, andtensor.random shape- overloaded
+,-,*,/,^, and unaryneg shape,reshape,transpose,tensorAt,square,tensorSum,mean, andmatmulconv2d,maxPool2d, andavgPool2dfor NCHW tensorsavailableDevices,deviceOf,dtypeOf,toDevice,toDtype,tensorSlice,assignTensor,hostMaterialized?,materializeTensor,tensorData,tensorBuffer, andtensorFromBuffer
Tensors contain floating-point data, inferred shape, row-major strides, and an explicit float32 or float64 dtype. tensor.dtype: selects identity, dtypeOf inspects it, and toDtype creates an owned conversion. Mixed binary operations promote to float64 when either operand is float64. Rectangular nested blocks may have any rank, and broadcasting aligns trailing dimensions. tensorSum.axis: and mean.axis: reduce one explicit axis (negative axes count from the end); without axis: they reduce the whole tensor. transpose.axes: accepts a full axis permutation, while plain transpose swaps the last two axes. tensorAt value [i j ...] returns an owned scalar copy and supports negative indices.
tensorSlice.axis:.start:.count:.step:, reshape, and transpose create lazy views. Views retain their source's device and dtype; hostMaterialized? stays false and tensorData (or any tensor operator) materializes an owned, versioned projection on demand, while eager arithmetic returns an ordinary owned tensor. Shape views compose as tagged projections: a reshape preserves flat order and a transpose permutes it. Device views retain the root allocation, so the source may be released first. Backends may implement native slicing, reshaping, and transposing; MPS uses slice/gather, buffer-sharing reshape, and transpose, and can feed a projected view directly into a compiled program without filling its host cache.
assignTensor target replacement mutates an owned tensor or the root selected by a view. Scalars fill the target; tensor replacements require matching shape and dtype. Replacement values are snapshotted before mutation, making overlapping slice assignments deterministic. Alias caches are versioned and invalidated lazily. Device writes atomically swap the shared handle before releasing the old allocation; backends with a scatter capability (MPS) realize the replacement entirely on the device, and portable backends fall back to a host copy-and-upload.
matmul follows the usual generalized rules: two vectors produce a scalar dot product, matrix–vector and vector–matrix products remove the promoted unit dimension, and rank-2-or-higher operands broadcast their leading batch dimensions. The same shapes and batch accumulation rules apply to reverse-mode gradients and scheduled CPU execution.
conv2d, maxPool2d, and avgPool2d participate in reverse-mode differentiation, including overlapping windows, stride, and padding. Convolution produces gradients for both NCHW inputs and OIHW weights; max pooling routes each window to its first maximum, while average pooling divides across valid, unpadded elements.
Every tensor carries an explicit device and dtype. CPU supports float32 and float64; importing src/metal-backend.art adds the native float32 MPS device on supported Apple silicon. registerBackend, backendAvailable?, and availableDevices expose the registry, while releaseTensor explicitly frees owned non-CPU storage. Device and dtype metadata survive graph export, optimization, and scheduled CPU execution.
Compiled device results are lazy on the host. hostMaterialized? distinguishes a resident-only result from one with a cached host mirror, materializeTensor fills that mirror once, and tensorData returns an owned host copy. tensorBuffer, eager CPU-style operations, and cross-device transfers materialize when their semantics require values. Shape and device inspection, release, and feeding an MPS result into another MPS executable do not download it.
variable, backward, gradient, detach, and zeroGrad form the autograd vocabulary. Scalar values are promoted automatically, so the smallest proof starts with x: variable 2.0.
A graph evaluates its labeled block once to establish the operation DAG. input declares a scalar placeholder, inputShape [1 4] bootstraps operations that require a matrix shape, and forward.with: supplies its runtime value. Graph fields are available dynamically (model\loss, model\w). inputs, parameters, and operations inspect its roles, while graphData returns a plain dictionary containing the graph order, nodes, shapes, operations, and parent relationships. explain renders that data for people.
sgd.rate: 0.05 (parameters model) creates a stateful optimizer. step updates parameter tensor data while preserving graph identity.
stateDict model returns an owned, named parameter snapshot built from tensor buffers. loadStateDict model state validates the complete snapshot before mutating anything and is strict by default; use loadStateDict.strict: false for intentional partial restoration. saveCheckpoint model path and loadCheckpoint model path round-trip the same state through a portable Arturo-text format that is parsed as data rather than executed.
optimizerState and loadOptimizerState preserve SGD rates, Momentum velocities, and Adam hyperparameters, moments, and timestep. saveTrainingCheckpoint model optimizer path and loadTrainingCheckpoint model optimizer path combine model and optimizer state; both halves are fully validated before either live object is changed.
batcher.size:.seed: creates a deterministic paired feature/target iterator. nextBatch returns owned batch tensors, original row indices, and the current epoch. batcherState restores the exact permutation and cursor, while saveSessionCheckpoint and loadSessionCheckpoint atomically combine model, optimizer, and data-order state.
train.epochs: consumes that iterator and returns learn.training-history with weighted training loss, validation loss, batch counts, and partial-epoch metadata. Use train.inputs: ['features 'labels] and train.loss: 'objective for custom graph names, train.validation: #[features: xValid targets: yValid] for validation passes, and train.callback: 'functionName to stop when an epoch callback returns false. A completed epoch leaves the batcher positioned at the start of the next epoch, so session checkpoints remain directly resumable.
gradientNorm measures the global L2 norm across parameter gradients, and clipGradNorm.max: rescales them in place when that norm exceeds the limit. stepLr.every:.factor: creates an epoch scheduler; pass it with train.scheduler: and optionally enable clipping with train.clip:. schedulerState and loadSchedulerState expose ordinary state dictionaries. Supplying the scheduler to saveSessionCheckpoint.scheduler: and loadSessionCheckpoint.scheduler: extends the atomic session checkpoint to its progress and current optimizer rate.
fit.linear features targets and fit.logistic features targets build and train ordinary learn graphs. Use predict, predict.probability, score, mse, and accuracy to evaluate them. split.ratio: performs a deterministic paired row split. The first estimator layer accepts vectors or one-column matrices.
fit.mlp.hidden: 4 features targets builds a dense one-hidden-layer regression graph. relu, tensorTanh, dense, momentum, and adam are also public building blocks.
standardScaler trainingFeatures fits feature-wise means and population standard deviations across the leading row axis. applyTransform scaler values applies the frozen statistics without mutation, and inverseTransform reconstructs the original scale. Rank-1 inputs are treated as one feature; higher-rank inputs preserve their complete trailing feature shape.
transformPipeline @[firstStep secondStep] composes fitted preprocessors in order and reverses them in reverse order. preprocessorState and preprocessorFromState provide owned ordinary-data state, while savePreprocessor and loadPreprocessor use a portable, non-executable v0.16 checkpoint payload.
softmax logits normalizes the final class axis with max-shifted exponentials. crossEntropy logits classIndices computes stable mean categorical cross entropy directly from a class vector or sample-by-class matrix and has the fused (probability - target) / samples gradient. Both operations participate in eager graphs and scheduled CPU execution.
classPrediction logits returns argmax class indices with the class axis removed. multiclassAccuracy targets predictions compares integer-valued class tensors. inputShape makes matrix classifiers declarable before their runtime batch shape is known.
dropout.rate:.seed: creates deterministic inverted dropout for tensors or graph values. Graphs begin in training mode; trainMode, evalMode, and trainingMode? control dropout explicitly. Evaluation mode is an identity operation, while training forward passes advance a seeded mask sequence and backward reuses the exact mask from its forward pass.
l1Penalty.rate: and l2Penalty.rate: return differentiable scalar penalties for a non-empty parameter block. Add them to an ordinary data loss inside a graph. regularizationState and loadRegularizationState preserve dropout counters and mode; v0.18 session checkpoints include that state automatically, with or without a learning-rate scheduler.
denseLayer.name: "encoder" inputSize outputSize creates a reusable weight/bias pair, and applyLayer layer input inserts a first-class differentiable dense node. A graph discovers reachable layer parameters even when they are not repeated as graph labels, deduplicates shared layers by identity, and exposes stable names such as encoder_weight and encoder_bias in graph data and checkpoints.
parameterGroups model groups parameters by layer name; parameters.group: selects a group and trainableParameters filters frozen values. Use freezeParameters/unfreezeParameters for a layer, graph, or parameter block and freezeGroup/unfreezeGroup for graph-owned groups. Optimizers retain frozen slots but skip updates. trainabilityState is included automatically in v0.19 session checkpoints.
initializer.seed: constructs a local deterministic xavier, he, uniform, or zeros strategy. initializeTensor spec shape never touches Arturo's global random stream. Initializer state is ordinary v0.20 data and round-trips through initializerState/initializerFromState.
Dense layers use Xavier initialization with seed 0 by default; pass denseLayer.initializer: with explicit, distinct seeds when same-shaped layers need independent streams. resetParameter, resetLayer, and resetParameters regenerate values from a strategy, clear stale gradients, and retain initializer metadata.
confusionMatrix actual predicted returns an actual-row/predicted-column tensor. classificationReport adds accuracy, macro precision/recall/F1, and per-class support, true/false positives, false negatives, precision, recall, and F1. Use .classes: when validation data may omit a class; absent classes receive zero-valued ratios.
confusionMeter classes accumulates batches with updateConfusion. mergeConfusion combines validation shards, confusionReport produces the same report, and confusionState/confusionFromState provide owned v0.21 state for distributed or interrupted evaluation.
compileInference model 'output snapshots the chosen output's optimized graph and parameter values in evaluation mode, pruning labels and other training-only branches. Add .preprocessor: fittedTransform.input: 'x to freeze preprocessing onto one named feed. runInference.with: executes the immutable artifact through the reference CPU schedule.
saveInference and loadInference package graph architecture, parameters, selected output, and optional preprocessing behind header 10022. A recursive tagged codec preserves strings, literals, numbers, logicals, nulls, blocks, and dictionaries as parsed data; artifact files are never evaluated as Arturo code.
inputGradient.input:.output: differentiates a named scalar graph output with respect to a named input feed; add .index: for one element of a vector or matrix output. saliency returns its absolute value. Both run in evaluation mode and restore the graph's prior mode and gradients afterward.
integratedGradients.steps:.baseline: averages those gradients along the straight-line path from a zero or supplied baseline and multiplies by the input delta. It uses the same named input/output and indexed-output attributes and leaves training state intact.
validateGraphData, deadCodeEliminate, commonSubexpressions, and optimizeGraph analyze or rewrite the plain dictionary returned by graphData. graphDot exports the same representation for Graphviz visualization. These passes preserve the live eager graph and make lazy execution work explicit at the data boundary.
scheduleGraph data 'loss applies constant folding, alias-preserving CSE, output pruning, and elementwise fusion planning. executeCpu.with: feeds schedule evaluates that immutable parameter snapshot using the pure-Arturo CPU reference backend.
compileNativeCpu schedule lowers eligible fusion groups to generated C when clang is available. executeNativeCpu.with: feeds artifact uses those kernels and automatically falls back to the reference backend for unsupported groups or dynamic shapes.
On macOS, build the vendored backend once with cd vendor/metal && arturo scripts/build.art. Import src/metal-backend.art, then use compileMps schedule and executeMps.with: feeds artifact for reusable compiled MPSGraph execution. Set LEARN_METAL_LIBRARY to override the dylib path. Dense math, broadcasting, reductions, activations, softmax, dropout, cross entropy, reshape, transpose, and view slice/gather/scatter are lowered without CPU operator dispatch; releaseMps frees the compiled artifact.
For a generic graph, import src/metal-autodiff.art and compile the same model and optimizer used by eager training:
optimizer: momentum.rate: 0.01.beta: 0.9 (parameters model)
artifact: compileTraining model 'loss optimizer
result: executeMpsTraining.with: #[x: features y: targets] artifact
loss: mpsTrainingLoss result
releaseTensor result
The pipeline is ordinary data: differentiateGraph graphData 'loss appends automatic-gradient nodes, and optimizerGraph differentiated optimizer appends parameter and optimizer-state updates. compileTraining performs those transformations and lowers the complete graph to one reusable MPSGraph executable. Parameters and momentum slots remain on the GPU; the returned loss tensor is lazy until mpsTrainingLoss, tensorData, or another host boundary is used. mpsTrainingModelState and mpsTrainingOptimizerState remain explicit checkpoint materialization boundaries.
Import src/metal-training.art and create mpsMlpTrainer.rate:.seed: batch input hidden classes. Its compiled program performs the dense forward pass, cross-entropy backward pass, ReLU gradient, SGD update, and next-step parameter handoff on MPSGraph. Only the scalar loss is downloaded during training; mpsMlpState is an explicit checkpoint boundary.
loadMpsMnist maps raw IDX bytes directly into normalized float32 Metal tensors. mpsMnistBatch slices batches on the GPU, trainMpsMnist.epochs: trains them, and evaluateMpsMnist evaluates the test partition. The included one-epoch 784–128–10 example reached 94.57% test accuracy in 4.35 seconds on the development Apple M4 Pro; results vary by machine.
./scripts/download-mnist.sh
arturo examples/mnist-mps.artThe downloader uses the CVDF MNIST mirror and validates the four gzip archives before extraction.
The package requires Arturo 0.10.0 and uses unitt:
arturo -p install unitt
~/.arturo/packages/bin/unitt --no-color
arturo examples/linear.art
arturo examples/regression.art
arturo examples/training-loop.art
arturo examples/preprocessing.art
arturo examples/multiclass.art
arturo examples/regularization.art
arturo examples/composition.art
arturo examples/initialization.art
arturo examples/metrics.art
arturo examples/inference.art
arturo examples/explainability.art
arturo examples/native-cpu.art
arturo examples/compiled-mps-training.art
arturo examples/mnist-mps.artThe MPS adapter is optional and the default src/learn.art entry remains CPU-only and portable. Backend-native scatter assignment and zero-copy reshape/transpose views remain outside the current milestone.