Repository navigation
contributor examples
Use Python 3.11/3.12 and Node 20.9+. Run python3 scripts/start.py from the repository root, or reopen the repository in its development container. The container installs dependencies without downloading weights; launch the model only when needed. For frontend-only work, run npm ci && npm run dev in frontend/, then choose Try a recorded demo.
- Copy
examples/contributors/TokenProbabilityPanel.tsxtofrontend/components/ui/TokenProbabilityPanel.tsx. It reads actual recorded top-k probabilities, preserves their original normalization, and provides an empty state. - Add
{ id: "token_probability", label: "Token probabilities", purpose: "Inspect captured next-token probabilities." }to the ANALYSIS tools infrontend/lib/debuggerTools.ts. - Import the component in
frontend/components/ui/DebuggerPane.tsxand addtoken_probability: <TokenProbabilityPanel />toviews. - Open the recorded demo, then Debugger → Token probabilities. Confirm values match
frontend/public/demo/hello-world.json, that empty state works before loading, and that navigating away unmounts the panel. - Run
npm run test:unitandnpm run buildinfrontend/.
Keep inference outside React components. Declare missing measurements rather than deriving fake attention, timing, or causal evidence from unrelated values. Shared scene labels use SceneHtml; passive labels are collision-filtered and remain below dialogs.
Run the small example:
PYTHONPATH=backend backend/.venv/bin/python examples/contributors/model_adapter.pyCopy its class to backend/app/inference/adapters/<family>.py, give it an exact model_type match, and add an instance to SPECIFIC_ADAPTERS in that directory's __init__.py. Register before the generic fallback. Add positive and negative config fixtures to backend/tests/test_capabilities.py.
A capability adapter describes support; it does not implement a new inference architecture. Before enabling a capability, implement and verify the corresponding runtime module layout in the engine/instrumentation adapters. Test against a real small checkpoint: compare attention to an independent forward pass, check layer/head shapes, ensure hooks are removed after errors, and demonstrate that an intervention changes the measured output. Keep unverified capabilities disabled with a reason, as the example does.
npm run test:unit covers pure logic; npm run test:store covers streaming state. backend/.venv/bin/python -m pytest backend/tests runs backend tests (install pytest first). The browser tests require a frontend on port 3000; first-use tests deliberately block the backend. A PR should state the user-visible behavior, data source, and actual checks run. See beginner tasks for current starting points.
TokenPrint Wiki | Home | Getting Started | User Guide | Concepts | Architecture | API | GitHub
TokenPrint is an interactive 3D visualization platform for exploring transformer architectures, tensors, and real-time LLM inference.
- User Guide
- Architecture Explorer
- Live Inference
- Tensor Inspector
- Camera Controls
- HUD
- Timeline
- Scene Navigation
- Transformer Concepts
- Tokenization
- Embeddings
- Positional Encoding
- RoPE
- Multi-Head Attention
- KV Cache
- LayerNorm
- Feed Forward Network
- Residual Connections
- Logits
- Softmax
- Sampling
- Autoregressive Generation
- Visualization System
- Scene Graph
- Geometry
- Materials
- Animation System
- Color Mapping
- Camera System
- Interaction Model
- Architecture
- Overall Architecture
- Frontend
- Backend
- WebSocket Protocol
- Data Pipeline
- Renderer
- Event System