Auto-Regressive Generation Loop

The engine is no longer a one-shot calculator — it now generates text token-by-token,
streaming results live to the browser.

What's New

Auto-Regressive Pipeline

• forward(tokenId) — Single-token forward pass through the full Layer 0 transformer
  (Embedding → RMSNorm → Self-Attention → Residual → RMSNorm → SwiGLU MLP → Residual → RMSNorm → LM Head).
  Advances the KV cache automatically on each call.
• generateText(prompt, maxTokens, onToken) — Two-phase generation:
  • Prefill: tokenizes the prompt and feeds every token through forward() to warm the KV cache
  • Decode: greedy argmax loop with EOS detection (Llama 3 tokens 128001, 128009) and
    KV cache bounds checking (MAX_SEQ_LEN = 128)
• getEmbeddingByTokenId(tokenId) — Direct embedding lookup by raw token ID, bypassing
  the tokenizer (used internally by the decode loop)

Streaming UI

• Button renamed Compute → Generate
• Prompt echoed in gray, generated tokens stream in green as they arrive
• Final stats show total time and average ms/token (including prefill)

Performance (Layer 0 only, greedy argmax)

Device	ms/token (avg)	20 tokens total
MacBook M2 Max	44.4 ms	~1.0 s

Known Limitations

• Single layer (1/26) — repetitive output is expected; the model lacks the depth
  to produce coherent multi-token sequences. This is not a loop bug.
• Greedy decoding only — no temperature, top-k, or top-p sampling yet.
• Next milestone: multi-layer stacking to unlock real generation quality.