Longer, complete examples of Soma — the kind that carry one problem all the way through instead of demonstrating one feature at a time.
The tutorial notebooks in the main repository teach Soma piece by piece. This repository is the other half: each example here is a real piece of work, and every diagram, figure and number in the READMEs was produced by running the code in it.
git clone https://github.com/manucouto1/soma-examples && cd soma-examples
pip install -r requirements.txt
python 01_pipeline_end_to_end/run.pyExamples 01, 02, 04 and 05 need nothing but the requirements. Example 03 needs a real model — there is no mock in this repository:
export OLLAMA_HOST=http://your-ollama:11434 # or any OpenAI-compatible providerNothing downloads a dataset. sklearn.datasets.load_digits ships inside
scikit-learn — 1797 real handwritten digits — because an example that
fetches from the network is an example that breaks when a URL moves.
Note on the Soma dependency.
requirements.txtcurrently installs Soma from git. It moves tosomatizefrom PyPI once the first release is published, at which point this repository's CI doubles as the proof that the published package works for somebody who is not its author.
Four things that were not obvious to the author of these examples, and that every example below now shows rather than assumes.
A plain Filter learns state in fit and transforms in forward. A
DifferentiableFilter carries _differentiable = True, which the bridge
reads when the node is registered, and that one attribute changes what the
compiler produces:
g.compile(mode="differentiable")["plan_text"]
# Composite[encoder → head]Consecutive differentiable nodes collapse into one Composite block.
That is what lets a single backward cross the boundary between nodes: to
autograd there are not two nodes, there is one graph. Put a
non-differentiable filter between them and they stop collapsing — the
gradient really does stop there, and the compiler says so.
Most preprocessing pipelines are not differentiable, and that is correct. Example 01 has one of each, and keeps them apart.
build_module(input_shape) describes how to build; nothing is built
until someone says how wide the input is:
enc = TwoView()
enc._module # None
g.materialize(x) # now it exists
# 11,506 parametersThat is what lets you write nn.Linear(input_shape[-1], 32) without
knowing what the node above produces. Each node's input shape comes from
its predecessors — every root gets the graph's input, and a fan-in node
builds lazily on first forward because it is the only thing that knows how
it combines its inputs.
This one costs you a model if you meet it the hard way.
for step in ...: # weights change in the live torch module
...
g.backward(ctx, loss); g.step(ctx)
g.state() # {} — the graph knows nothing
g.save("model.somack") # 395 bytes: the topology, and no weights
Graph.load("model.somack") # different predictions. The training is gone.The missing step is g.freeze(), which snapshots every live module into
the graph's own state:
g.freeze()
g.state() # {'encoder': ['weights_b64'], 'head': [...]}
g.save("model.somack") # 43,787 bytes
Graph.load("model.somack") # identical predictions ✓It is documented — design/gradients
and the Python API reference both cover it — but it is easy to miss,
because eval() and forward() keep working without it in the process
that did the training: the live module is still there. You find out when
you save, or when the node travels to a worker.
The whole cycle:
materialize(x) build each module, sized from its predecessors
train() / step() weights move, in the live module
freeze() module → state, mark fitted, switch to eval
save(path) persist; Graph.load(path) restores it identically
g.to_svg(overlay=g.architecture_overlay(flags=audit.report()))
# encoder["encoder<br/>⚠ LEAKAGE · ⚠ DEAD_CHANNELS · 11.5k θ"]g.architecture() answers the same question as data — module class,
trainable and frozen counts, built or not, per node and in total — and
architecture_overlay() shapes it for the renderers, which all already
accepted an overlay. A graph of plain Filters annotates nothing and renders
exactly as before.
Every graph renders itself — to_text(), to_svg(), to_mermaid() — and
after a tracked run, RunView renders the same graph annotated with what
happened: durations, cache hits, health flags. Every example here writes
both, so the diagrams in the READMEs are the architecture that ran, not a
drawing of it.
| # | Example | What it shows |
|---|---|---|
| 01 | End to end | The DSL, the compiled plan, profiling, cache reuse, a gradient audit that finds three injected pathologies, and a TPE search. 0.238 → 0.781 → 0.950 |
| 02 | Experiment campaign | An ablation whose answer the obvious method gets wrong: lineage, checkout/diff, a conclusion recorded on the dead end, retrieval by words and by shape. Interaction +0.062 |
| 03 | Agentic research | No mock — a real model, a tool that reads the pool, a judge that scores grounded vs ungrounded (0.5 vs 0.0), and a panel that votes |
| 04 | Streaming | 20k samples in ten chunks, asserted not narrated: stream == batch, a re-run does no work, editing one sample re-runs 1 of 10 chunks, a barrier sees everything |
| 05 | Distributed placement | A real worker, target= per node, local == remote. Found and fixed a worker bug on its first run; documents exactly where the DataStore path stops |
| 06 | Multi-view ensemble | No mock. Real LLM embeddings + two other views, one predictor each, a learned gate. Every branch gets gradient; the gate's ranking matches the solo accuracies exactly |
Soma's TrainingStrategy (DataParallel, Federated, ModelParallel,
PopulationBased) is a graph attribute that nothing currently reads: the
training loops exist in soma-runtime and have no caller, and Python has no
set_strategy. Example 05 therefore demonstrates placement — which does
work — and not gradient synchronisation, which does not. The upstream
Execution Modes
page says the same.
common/ the dataset and the model every example shares
NN_name/ run.py + README.md, one story each
artifacts/NN/ what run.py produced: SVG, PNG, interactive HTML, CSV
Artifacts are committed. They are the point of reading the README without running anything, and they are how a change that breaks a figure becomes visible in a diff.