TaleWeaver v0.5.0-beta

[jschm42]

TaleWeaver Release v0.5.0-beta 🚀

Welcome to TaleWeaver v0.5.0-beta! This release marks a significant milestone, shifting TaleWeaver from a text-and-audio game to a fully immersive generative RPG platform with local speech processing, a fully functional world builder, and story expansion features.

🎧 Speech-to-Text & Push-to-Talk (PTT)

Experience true hands-free roleplaying. Speak directly to your AI Gamemaster or converse in-character with NPCs.

• OpenAI Whisper Integration: Support for local Whisper model execution.
• Background Preloading: Whisper models load in a separate background thread to keep the user interface fully responsive, displaying real-time loading states.
• Push-to-Talk: Press and hold UI mic buttons or keys (V, B, or T) to capture audio, with instant auto-transcription and submission upon release.
• In-Character Speech Shortcuts: Hold down B, T, or Shift to automatically prepend /say to your message, seamlessly shifting between gameplay actions and direct spoken dialogue.

🛠️ Fully Functional Adventure Editor

You can now create, edit, and build complete adventure templates from scratch or customize existing ones directly in the app.

• Full CRUD Management: Create, read, update, and delete scenes, exits, quests, NPCs, items, containers, and switch outcomes.
• Bidirectional Route Mapping: Exit creation handles bidirectional linking automatically, storing connections cleanly in a single database row.
• Asset Integration: Associate custom visual assets, set styles, and refine details of all generated entities.

🌀 Cover Mode (Sequels & Variations)

Expand your library by generating sequels or spinoffs of your favorite adventures.

• Similarity Index: Set a similarity index (from 0% for freely inspired variants to 100% for close sequels).
• Asset Portability: Toggle the reuse of visual styles and media assets from the original cover source to keep the look-and-feel consistent across generations.

⚙️ Core Enhancements & Stability

• Chronicles Checkpoints & Timeline Rollback: Automatically saves your state at major milestones (scene entry, quest updates, achievement unlocks). The new Chronicles timeline allows you to rewind time and roll back the story.
• Interactive World Map: Fully interactive map with pathfinding, panning, and zoom support to visualize connections between your scenes.
• Path Traversal Security Hardening: Hardened path handling routines across media routes and APIs to mitigate uncontrolled file system risks.
• Robust Serialization & Import/Export: Excluded system columns (pk, session_id, created_at, updated_at, template_id) from exports. This fixes sqlite JSON serialization errors (datetime objects not serializable) during adventure template imports.
• Model Integration: Support for new LLM providers like Kimi (Moonshot), as well as automatic LLM blank JSON retry logic to maximize parsing success rates.

---

🧪 Installation & Running

Backend

Make sure to apply database migrations before launching:

# Activate virtual environment
venv\Scripts\activate  # Windows
source venv/bin/activate  # macOS/Linux
# Apply migrations
python -m alembic upgrade head
# Start FastAPI server
python -m backend.main
Frontend
bash
cd frontend
npm install
npm run dev
For Docker users, run bash scripts/docker-update.sh to fetch and apply the latest image updates.

<hr /><em>This discussion was created from the release <a href='https://github.com/jschm42/taleweaver/releases/tag/v0.5.0-beta'>TaleWeaver v0.5.0-beta</a>.</em>

[richardchen874-sys]

This is an interesting release, especially the move from text/audio toward a more complete generative RPG platform.

For AI-driven RPG systems, I think the hard part is not only generating good content once, but keeping the whole gameplay loop stable across many turns: GM narration, NPC dialogue, scene state, quests, generated assets, and rollback.

The Kimi / Moonshot provider support and blank JSON retry logic are especially important signals. In game workflows, malformed JSON or empty model output is not just a formatting issue — it can break scene transitions, quest state, NPC behavior, or import/export flows.

A few metrics that may be useful to track over time:

• cost per completed scene or quest step
• failed / blank JSON rate by provider
• retry count per gameplay action
• latency for GM narration vs NPC dialogue
• whether different providers handle structured outputs differently
• consistency of generated sequels / variations
• asset generation failure rate
• context growth across long play sessions
• rollback frequency after bad model output

For generative games, provider choice is less about one model being “best” globally and more about which model stays reliable for each part of the workflow: narration, dialogue, structured state updates, asset prompts, and recovery after failures.

It would be interesting to see future notes comparing provider behavior across those gameplay tasks.

[jschm42]

Thanks for this interesting post. Indeed these are some important metrics to find the best models for the job. I might add some auto collection mechanism to collect some of the data. In the end a table comparison with the different models would be great.

My personal favorite ist curently DeepSeek-v4-pro for the naration and DeepSeel-v4-flash for the rule pass. This ist very cost efficient and generates very good output. For the world building, it's best to use a flagship model from Anthropic or OpenAI. I use mostly GPT 5.5. And I can recomend to use local Black Forrest Labs Small/Dev Models for Image-Generation. They generate fantastic outputs.

A few notes on how some of these map to what's in the engine today, and where we're heading:

On the stable gameplay loop

This is the core reason TaleWeaver runs a 2-pass pipeline per turn rather than a single narration call:

• Pass 1 (Mechanics): a strict, schema-validated rules pass that computes state changes: HP/stamina/mana, quest deltas, inventory mutations, scene transitions, combat resolution.
• Pass 2 (Narrative): takes the already-resolved state from Pass 1 and only produces prose + TTS.

The key design decision is that the narrative model is never the source of truth for game state. If narration drifts or hallucinates, the underlying turn state is still correct. That separation is what keeps the loop from compounding errors across many turns — the exact failure mode you're describing.

On per-task provider selection.

Your closing point - that "best model globally" is the wrong question - is precisely the philosophy behind our LLM Recommendations table. We already let users pick different providers/models for World Generation vs. Mechanics (Pass 1) vs. Narrative (Pass 2), because the failure modes genuinely differ:

• World generation needs deep, valid JSON: heavy reasoning models (GPT-5, Claude 4.5 Opus, Gemini 3 Pro).
• Mechanics needs strict rule-following: mid-tier but reliable (GPT-5-mini, Claude 4.5 Sonnet. deepseek-v4-flash).
• Narrative wants prose quality + low latency for the audiobook feel.

This is also why Kimi/Moonshot support matters: more providers = more levers to tune each pass independently, and a hedge against any single provider's structured-output quirks.

On the blank JSON problem. We feel this one viscerally. The blank-JSON retry logic exists because empty/malformed output in Pass 1 doesn't just fail a turn - it can desync quest state or block a scene transition. Right now recovery is automatic retry; the Chronicles timeline + rollback is the user-facing safety net when retries are exhausted. We cap checkpoints to the last 5 to bound context bloat, which ties directly into your "context growth across long play sessions" metric.

Thanks for your great input.
Jean

[richardchen874-sys]

@jschm42 This is a very helpful breakdown

The 2-pass design is exactly the kind of separation I was hoping existed under the hood.

Making the Mechanics pass the source of truth, and keeping Narrative as prose/TTS only, seems like the right safety boundary for a generative RPG. If the narrative model drifts, the game can still preserve the real state because HP, inventory, quest deltas, scene transitions, and combat resolution already came from the validated rules pass.

The per-task provider selection also makes a lot of sense:

• World Generation: deep JSON / stronger reasoning
• Mechanics Pass: strict schema and rule following
• Narrative Pass: prose quality and low latency
• Image Generation: local image models when possible

That is a much better framing than trying to find one “best” model for the whole game.

If you add auto collection later, I think per-pass metrics would be very useful:

• Mechanics JSON failure / retry rate
• Narrative latency
• rule-pass validation failures
• rollback frequency
• provider used per pass
• cost per completed turn
• context size per session
• checkpoint growth / cleanup behavior
• scene transition failure rate
• whether a failure came from model output, schema validation, or state logic

I’m also working around managed OpenAI-compatible endpoints and multi-model API workflows, so this kind of per-task routing is especially interesting to me.

For generative games, I think the strongest architecture is exactly what you described: not one global model choice, but a workflow where each pass can use the model that best fits its failure mode, cost profile, and latency requirement.

[jschm42]

Yes, it's working better than expected. Although there are still a lot of problems that need to be solved. Good point collecting those metrics. Will put this on the roadmap.

[richardchen874-sys]

@jschm42 , this architecture makes a lot of sense.

Separating Mechanics from Narrative is probably the right way to avoid the worst failure mode in AI RPGs: letting a nice-sounding narration accidentally become the source of truth for game state. The 2-pass design also makes model evaluation much cleaner, because you can judge each provider by the task it is actually responsible for.

Your model split is interesting too:

• DeepSeek-v4-flash for rule / mechanics pass
• DeepSeek-v4-pro for narration
• GPT-5.5 / Claude / Gemini-style flagship models for world generation

That feels very aligned with the idea that “best model” should be measured per workflow step, not globally.

If you add auto collection later, one table I would personally find useful is something like:

• pass type: world generation / mechanics / narrative
• model/provider
• average latency
• blank or malformed JSON rate
• retry rate
• cost per completed turn
• state correction / rollback rate
• subjective output quality for narration

For TaleWeaver specifically, I think “cost per stable turn” may be more meaningful than raw token price, because a cheaper model that causes retries or rollbacks can become more expensive in the actual gameplay loop.

I’m also working on an OpenAI-compatible multi-model API layer, so this kind of per-task routing problem is very relevant to me. Curious: for your current LLM Recommendations table, are you planning to make it mostly based on your own testing, or will TaleWeaver eventually collect anonymous user-side performance data across different providers?