Architecture

TensorPlay is organized as four deliberately decoupled libraries that can work together or independently. This page maps the stack bottom-up and points to the deep-dive material for each layer.

The four pillars

Component	Role
tensorplay	The Python API: tensors, autograd, nn, optim, data loading, serialization — the public surface
p10	C++ core engine: tensor storage and memory management, foundational CPU and CUDA kernels
tpx	Autograd layer: explicit computation-graph construction and backward execution, fully decoupled from the core
stax	JIT compiler playground: static graph capture and operator fusion experiments, including native lowering of custom ops

Key design points

• Explicit computation graphs. Autograd is not hidden behind an implicit recorder: TPX builds a visible DAG, and the backward pass replays the chain rule node by node — every edge is code you can step through.
• A thin, measurable dispatcher. Operations route through a 13-slot dispatch table with static registration; measured overhead is below 1% on CPU, CUDA, and Vulkan paths.
• Transparent storage. DataPtr / Storage / TensorImpl separation makes memory ownership explicit, and the 7-step TensorIterator pipeline handles dtype/broadcast/stride logic in reviewable stages.
• Custom hardware friendly. The Vulkan teaching backend (145 ops over 4.5k lines of GLSL) exists as a worked example for bringing up new device backends.

Where to dig deeper

Resource	What it covers
White paper (PDF)	The whole stack with path:line citations: dispatcher, autograd engine, memory model, TensorIterator, codegen
Blog series	One pillar per post: four-pillars overview, tensor storage, dispatcher & codegen, TensorIterator, autograd engine, stax graph compiler
Module docs	API reference for nn, optim, linalg, fft, sparse, amp, and more
benchmark/	Reproducible end-to-end and per-op benchmarks with JSON reports

Benchmark highlights

Every comparison in the benchmark suite runs both runtimes from identical initial weights, deterministic data order, and the same optimizer and batch settings — with a strict correctness gate (logit match + identical Top-1 predictions) before any timing is reported. Highlights from the white paper evaluation:

• Dispatcher overhead: sub-1% on CPU, CUDA, and Vulkan paths
• CUDA backend: 1,274 unique ops (96% of the CPU surface)
• Vulkan teaching backend: 145 ops, 4.5k lines of GLSL shaders