Skip to content

MTPLX 2.12.3

Latest
Latest

Choose a tag to compare

View all tags
@youssofal youssofal released this 06 Nov 21:23
· 1 commit to main since this release
v2.12.3
15c8afd

MTPLX 2.12.3

MTPLX 2.12.3 is a bug-fix release. Its main fix brings back the prompt cache for Flash-Next in OpenCode and in plain API clients. In 2.12.2 those clients read their whole conversation again on every request, so each turn waited longer for its first token than the one before.

It also fixes an HTTP 500 that stopped Flash-Next turns after a compaction, a restore from the SSD cache that could build an answer on another conversation's saved state, four more SSD cache bugs, a server that stopped answering after about two days on an Ultra, and a set of problems in Claude Code, Codex and OpenCode sessions.

Highlights

What you do 2.12.2 2.12.3
Start an OpenCode 2.2.5 session on Flash-Next Optimized Speed (its first 21 requests) 2 refused with "already in flight". All 18 follow-ups read their whole prompt again (21,452 to 36,643 tokens each), so no prompt token came from the cache None refused. No follow-up read its prompt again. 97.2% of prompt tokens came from the cache, and the median first token took 0.28 s
Keep a plain chat going with the SSD cache off, from a client that drops reasoning (#291, with the reporter's flags) Turns 2 to 5 read their whole prompt again, with 0 tokens from the cache From turn 4, each turn resumes where the previous prompt ended (888 of 1,615, 1,615 of 1,955 and 1,955 of 2,006 tokens from the cache)
Run a five-turn OpenCode 1.19.5 session on Flash-Next 0 of 19 requests after the first reused the cache. The first token took 3.0 to 9.9 s and grew with the conversation (median 6.3 s in tool loops) 30 of 30 reused it, for 98.4% of all prompt tokens. The first token took 0.15 to 0.98 s (median 0.18 s in tool loops and 0.17 s at each new turn)
Run a five-turn Pi 1.4.0 session on Flash-Next 97.8% of prompt tokens from the cache, 34 of 34 requests 96.7%, 24 of 24. Pi was never affected by this bug
Run plain API chat, OpenCode chat, an OpenCode tool loop and a new chat about the same document, on a small Flash-Next test model 0 of 16 follow-up turns reused the cache 16 of 16, with the SSD cache on and off, and 12 of 12 with a 17,000-token first message
Continue a Flash-Next conversation after a compaction, with the SSD cache on (#592, #605) HTTP 500 "QSA pooled hold N rows; resizing to M would cut them" Served (a replay of #605, in tests)
Resume a Flash-Next conversation from the SSD cache when a longer saved conversation starts the same way (#635) 25 of 64 new tokens got a different attention output, taken from the other conversation (in a replay) The same tokens and final logits as reading the conversation from scratch (in tests)
Resume a 62,000-token Qwen 3.8 27B conversation from the SSD cache on an M5 Pro with 64 GB (#668) 124.7 s to the first token, decode 15.0 tok/s 23.4 s, decode 23.7 tok/s
Send a message from Codex desktop 26.1016 (#658) Refused with a 400 Served (the report's request, in tests)
Run several Claude Code subagents at once 409 "already in flight", then retries Each subagent runs in its own session (Claude Code's captured headers, in tests)
Keep a Flash-Next server running for two days on an Ultra (#640) Every request failed at Metal's 499,000-buffer limit until a restart Idle conversations release their saved state and the next request runs (in tests)

How these were measured:

  • The OpenCode 2.2.5 and #291 rows ran on November 4 on an M5 Max with 128 GB, through mtplx serve on Qwen 3.8 Flash-Next Optimized Speed with fans at maximum, 2.12.2 and 2.12.3 one after the other. The SSD cache was on, except in the #291 row.

  • OpenCode ran as the command-line client inside OpenCode Desktop 2.2.5, with MTPLX's own OpenCode setup. The 2.12.2 OpenCode run started after an hour of continuous work at heavy thermal pressure, so its first-token times are not compared. Refusals, re-reads and cache shares do not depend on heat.

  • The #291 row uses the reporter's flags (--profile turbo --context-window 131072, SSD cache off). Its turns 2 and 3 share fewer than 512 tokens with the turn before, which is below the size a saved prefix is restored from.

  • The OpenCode 1.19.5 and Pi 1.4.0 rows ran on October 30 on the same Mac, through mtplx serve on Qwen 3.8 Flash-Next Bare Speed with the SSD cache on and fans at maximum. Each client ran alone, with the same five prompts in a fresh project and a fresh home folder, 2.12.2 and 2.12.3 one after the other. The answers differ between runs (temperature 1.0), so the sessions do not grow to the same length.

  • Times to the first token are the server's own, from its request log.

  • The test-model row uses a 0.2 GB model with Flash-Next's structure, served by the real server on the same compiled path. It shows cache reuse, not speed.

  • The #668 row is the contributor's measurement. Rows marked "in tests" come from tests built from the reporters' requests, or from a replay of the reported failure.

What changed

Prompt cache

  • Flash-Next follow-up turns reuse the conversation again (#591, #616, #649). In 2.12.2, when MTPLX kept no live copy of a Flash-Next conversation, it saved the conversation without the draft head's history. That is the case for OpenCode's tool turns, for API clients that send no session id, and for a turn saved after it ended. The next turn found the saved conversation but could not resume from it, so it read the whole prompt again, and the request log called it a new session. Saved conversations now keep that history, as they did before 2.12.2. The conversation that stays live is still not copied. A turn that finds a conversation saved by 2.12.2 without its history now says no_snapshot_coverage instead of new_session.

  • A new conversation about the same document resumes inside the earlier prompt (#649, #637 by @jvmenen). When the memory guard skipped the separate save of a prompt read from scratch, the restore points recorded while it was read were dropped. The turn's final save now keeps them. A new conversation that shares the system prompt, the tools or a long document now resumes at the nearest of those points instead of reading everything again.

  • Plain Flash-Next chat with the SSD cache off resumes at the previous turn (#291). Some clients send the answer back without its reasoning, so the history changes where the assistant's turn starts. The next turn then resumes from a point saved at the end of the previous prompt. Without the SSD cache, that point was saved only for requests with tools, so every plain chat follow-up resumed at the first turn's prompt and read one more answer again than the turn before. Flash-Next now saves it for plain chat too, without copying the conversation.

  • A retry or regenerate resumes from the saved prompt (#661 by @a3rosol). Sending the same prompt again read it from the start: 23.5 s on a 16,000-token prompt on an M1 Ultra in the report. The saved answer now keeps the prompt's last logits, about 1 MB, so a retry starts answering at once.

  • Long sessions without a session id keep their cache (#624). Some clients send no session id on /v1/chat/completions. When MTPLX rendered one of their earlier answers back slightly differently from how it generated it, the session was lost once the conversation grew past about four times the point of that difference. The next turn then restored only an old part of the conversation: 35,440 of 163,242 tokens, and 95.6 s to the first token, in the report. A difference anywhere inside an answer MTPLX generated now keeps the session at any length. A difference inside a message you wrote still starts a new session.

  • MTPLX_MTP_HISTORY_POLICY=cycle keeps the prompt cache (#400, #422). On M1 and M2 Macs the default draft history can make each verify step slow (4 to 5 tok/s on an M2 Max in #400), and cycle is the setting that brings the speed back. With it, every turn read the whole conversation again: 166 s to the first token at 76,000 tokens and 292 s at 108,000 on an M2 Ultra, from #422. The server saved each turn under a different label from the one the next turn looked for. Saved conversations now restore under cycle as they do by default. Nothing changes if you do not set it.

  • Plain decoding resumes each conversation from the previous turn. With --generation-mode ar, --no-mtp, --stock-ar or --no-load-mtp on a Qwen model, a follow-up turn resumed from the last turn that added at least 1,024 tokens, so it could read up to about 1,000 tokens of the conversation again. Plain decoding now saves each prompt before it answers, as speculative decoding does, and the next turn resumes there. This matters most on M1 and M2 Macs that switch Flash-Next to plain decoding (#400).

Flash-Next and the SSD cache

  • No more HTTP 500 before the first token after a compaction (#592, #605). A turn failed with QSA pooled hold N rows; resizing to M would cut them, and instant retries failed the same way. It happened when a turn resumed from the start of a longer saved conversation: after a compaction, after an edited message, or in a new conversation with the same system prompt and tools. The attention index carried the longer conversation's block count into the restored turn. The count now comes from the restored conversation itself (#650 by @goodbadwolf), and a resize keeps only the blocks the conversation covers (adapted from #657 by @transcendence-x). In a replay of #605, the turn that raised the error now resizes 122,880 bytes of index buffers and continues on the compiled verifier.

  • No more answers built on another conversation's blocks (#635). The same inherited count made a restored turn pick attention blocks from the longer saved conversation, and every later turn inherited them. In a replay, 25 of 64 new tokens got a different attention output. A restored turn now produces the same tokens and final logits as reading the conversation from scratch.

  • Flash-Next conversations saved by older versions are read once more. Flash-Next conversations saved to the SSD cache by 2.11.2 to 2.12.2 may hold another conversation's blocks, and nothing in a saved entry says which, so they are no longer used. The first turn of each such conversation after updating reads its prompt once (about 92 s for a 65,000-token prompt on an M3 Ultra, from #635). The cache clears the old entries over time, as it does any entry, so nothing needs to be deleted by hand. Other models keep their saved conversations.

  • A conversation restored from the SSD cache runs on the memory layout its request asked for (#608 by @jvmenen, the same fix in #668 by @CocoaBeans). It always ran paged, so its first token and its whole answer were slower than after a restore from memory. Measured by @CocoaBeans on an M5 Pro with 64 GB and Qwen 3.8 27B Optimized Speed, a 62,000-token exact restore: prompt 50.1 to 303.2 tok/s, decode 15.0 to 23.7 tok/s, first token 124.7 to 23.4 s.

  • A conversation moved to the SSD cache restores its own state (#609 by @jvmenen). When another conversation with the same system prompt stayed in memory, the moved one restored only that system prompt (143 to 146 s at 50,000 tokens in the report, or a 507) and held a decoded 5.4 GB copy while its prompt was read. It now restores its own entry (about 4 s in the contributor's run), and candidates it does not use are released before the prompt is read.

  • A partial restore from the SSD cache works with paged caches (#655, fixed in #656 by @Phydra11). It failed with TypeError: int() ... NoneType on every retry, for example on an M1 Max with an 8-bit paged KV cache.

  • Flash-Next starts without its draft head again (#400). mtplx serve --generation-mode ar --no-load-mtp and --stock-ar refused to load Flash-Next with "relaxed Qwen4 draft ties require native MTP" (on an M1 Ultra in #400). Plain decoding is what you use on Macs where speculative decoding is slower.

  • A server at Metal's buffer limit keeps serving (#640). After about two days on an Ultra, every request failed with Resource limit (499000) exceeded until a restart. The memory guard freed bytes, but the limit counts buffers. The guard now releases the saved state of idle conversations, oldest and SSD-saved first. The 507 now says it is a buffer count, not memory, and that a restart recovers it if nothing idle was left. What still holds the buffers on those Ultras is not found yet (see Known issues).

Claude Code

  • Each conversation keeps its session through compaction, and each subagent gets its own (#606 by @jvmenen). MTPLX now reads Claude Code's session header. Subagents used to share the parent's session and got 409 "already in flight". A subagent now runs in its own session, and a second request on a busy Claude Code session runs beside it instead of failing.

  • Side requests no longer replace the conversation's cached turn. Claude Code also sends its session id with the session title, the auto-mode safety check and the command-prefix check. A request that does not share the conversation's first 4,096 tokens is now treated as a side request and gets its own cache. With CLAUDE_CODE_GATEWAY_HINT_HEADERS=1 (Claude Code 2.1.273 and later), MTPLX reads the request type Claude Code sends instead.

  • /effort is applied (#634 by @goodbadwolf). Claude Code's effort level was ignored. /effort low and medium now run at those levels, and max runs at xhigh. high, which Claude Code sends unless you pick another level, keeps the server's default (medium on Qwen 3.8 27B and Bonsai 2, xhigh on Flash-Next), so default Claude Code turns think as long as they did in 2.12.2.

  • The auto-mode safety check runs without thinking (#670 by @phaedrus1992, for #669). Claude Code asks it for a verdict on every tool call and sends no thinking setting, which on Anthropic's API means no thinking. Only Claude Code's exact safety-check request is matched: its billing block, the first sentence of its system prompt, no tools, no streaming and no thinking setting.

Codex

  • Codex desktop 26.1016 and its bundled CLI can send messages again (#658). Every turn was refused with a 400 because of reasoning_summary_delivery.

  • A preamble and the tool call after it are one assistant turn (#603). Codex sends a short preamble ("Reading the importer first:") as its own message before a tool call. MTPLX made two assistant turns of it. After 10 to 25 tool calls the model copied that pattern and stopped without a call, which Codex reads as the end of the task.

  • Codex keeps waiting while MTPLX reads a long prompt (#680 by @SuperMarioYL). The Responses stream sent nothing between its first status event and the first token. On a long prompt read from scratch, Codex's idle timer could end the turn before the answer started. MTPLX now sends a status event every 5 seconds until the first token. With Codex CLI 0.167.0, an 87,354-token prompt on Flash-Next and Codex's idle timer set to 10 seconds, the turn ended after 10 seconds without this change and completed in 64 seconds with it.

OpenCode and other agents

  • OpenCode 2.x title requests no longer stall your turn or throw away its cache. OpenCode Desktop 2.2 changed the prompt it uses to name a conversation, and sends it beside every turn under the conversation's own id. MTPLX no longer recognized it, so the title ran on the model. When the title reached the conversation first, your turn was refused with "already in flight" and retried, and the title replaced the conversation's saved state, so the retry read the conversation again (7 s to the first token instead of 0.2 to 0.6 s in a session on November 3, where 8 of 9 refused OpenCode requests were titles). MTPLX now answers OpenCode's title requests at once without the model, using the conversation's first message, so the title stays the same. MTPLX's OpenCode plugin also passes OpenCode's own request kind, so a title is recognized from the first turn of a new conversation.

  • Long OpenCode 2.x answers are no longer cut at 32,000 tokens. OpenCode 2.x sends its 32,000-token default with every request and no longer lets MTPLX's plugin remove it, so a long answer at xhigh stopped at 32,000 tokens. The server now drops exactly that default for OpenCode requests, as the plugin did. A cap you set with --max-response-tokens still applies.

  • MTPLX's OpenCode plugin works with OpenCode 2.x from the app too. OpenCode 2.x runs only a plugin's 2.x entry, which the app's plugin did not have, so under OpenCode Desktop 2.x the app's plugin did nothing. The app now installs the same plugin package as mtplx.

  • Clients that read the reply as text can keep tool-call markup (#674, #675 by @energypantry). On a request without tools, MTPLX replaces <tool_call> markup with a notice. That is right for the app's chat and wrong for a client with its own text protocol, where the markup is the command. The server's operator can turn the replacement off with MTPLX_KEEP_ORPHAN_TOOL_MARKUP=on, and a client can turn it off for its own requests with an x-mtplx-keep-tool-markup: on header or a "keep_orphan_tool_markup": true field. Measured by @energypantry on an M3 Ultra with Qwen 3.8 27B, on a 34-file refactor over a plain-text protocol: 36 turns and 452 s instead of 40 turns (the limit) and 577 to 696 s, and 1 lost command instead of 7.

  • The tool-call example shows how parameters are written (#584). In OpenCode's compact tool mode, the example call is the only place the prompt shows a call, and it was built from the first tool, which in the report takes no parameters. After a screenshot the model then wrote arguments as "<parameter=290>". The example now uses the first tool that has parameters, and is unchanged when the first tool has them. It has not yet been run with the reporter's script on a model.

  • An answer that ends inside an unclosed reasoning block is returned (#583, #588 by @jvmenen). With tools declared, it came back empty. It is now returned as the answer when the turn stopped on its own and holds no tool call.

The app

  • The Settings popover, the launch overlay and the model picker open in one step. They used to open in a stagger of about one second. They now open in a single 0.2 s step, or at once with reduced motion. On an M5 Max, in the 1.5 s after a click on Settings, the app used 550 ms of CPU instead of 881 ms and WindowServer 322 ms instead of 512 ms (median of six opens per version, the two versions taking turns). A slider you drag keeps its own value until you let go, then sends one change to the engine.

  • The app asks the engine for less while it answers. During any request, from Pi, OpenCode or the app's own chat, the app asks for a dashboard snapshot every 500 ms instead of every 100 ms, and once a second when its window is closed or hidden. The live gauge still moves 10 times a second, and a slower rate you chose in Settings is kept.

  • The gauge no longer costs a CPU core while the engine answers. The gauge's speed number and its glow were redrawn on the CPU for every frame of every digit change. They are now drawn on the GPU. On an M5 Max during a Flash-Next answer, with the Live tab showing, the app used 0.22 of a CPU core instead of 1.22 (216 against 1,215 ms of CPU a second). With the window closed it uses 0.004 of a core. The gauge looks the same.

  • The Live tab says when macOS slows the GPU because of heat. The notice shows while macOS reports heavy thermal pressure and clears when it eases, in all 13 languages.

Dashboard and request log

  • The request log says what really happened to a request (#652, #631, #600, #617 by @jvmenen). A request cancelled during decode keeps its real cache result instead of a miss. A request that found nothing to reuse says why (a new chat, no shared prefix, an evicted entry) instead of blaming the SSD cache. A 507 or 503 shows as a refusal, with its reason and when a retry can work. The flight log and the dashboard show how long a request waited for other work before it started.

  • The dashboard shows more, by @jvmenen. The Cache tab shows what RAM and the SSD hold (#622). The decode chart plots live samples over the whole five minutes and shows the value under the cursor (#598, #626). The verify waterfall is easier to read (#590).

Other fixes

  • Paste or drop images into the web chat (#601). The chat on the server's root page now sends a pasted screenshot or a dropped image file with your message when the model can read images. Each image shows as a thumbnail you can remove, and the chat says so when the model reads text only.

  • --scheduler-mode mtp_batch says which models it serves. It is a mode for the Qwen 3.6 35B-A3B packs. On Flash-Next its error asked for --verify-strategy target_prefix, which Flash-Next then replaced, so the same error came back whatever you set (a report on the Flash-Next model page). It now says the mode serves the 35B-A3B packs only, and which flags to drop.

  • DeepSeek MTP packs skip the output head when only the last row or no logits are needed (#636 by @sypsyp97).

  • The opt-in incremental SSD encode no longer mistakes a block with flipped signs for the old one (#596 by @jvmenen).

  • Structured output says why it is refused (#643). With llguidance installed but unable to load (1.3.0 in the report), every json_schema request said "pip install llguidance", which pip answers with "Requirement already satisfied". The refusal and mtplx doctor now name the installed version, the error, the 1.8 minimum and the pip command for the Python that MTPLX runs on.

  • Fan watchdogs exit when their fan lease ends (#602). In smart fan mode, each fan ramp started a watchdog process that stayed alive until the server stopped (87 of them after a day, 557 MB in all, in @CocoaBeans's report).

  • mtplx forge build works with a relative --out (#627, found and diagnosed by @Starefossen). The contract probe wrote its result one folder too deep, and calibration failed with agreement 0.0.

  • Gemma 4 loaded without its draft model runs (#632 by @jvmenen). Its first forward pass failed with unexpected keyword argument 'shared_kv'.

  • lock_wait_time_s in the request log includes the time a request waited behind another (#576 by @jvmenen).

  • The test suite ignores the shell's MTPLX_ settings and never opens the real session cache (#644 by @jvmenen).

Added

  • --trust-network serves without an API key on a network you trust (#491). A server bound beyond localhost still refuses to start without a key by default. --no-auth --trust-network together serve without one, and the startup banner says that anyone who can reach the port can use the model.

Dependencies

  • The lockfile has certifi 2026.10.18 (#672). The release workflows use newer GitHub Actions (#612, #613, #664, #665).

Validated

Check Result
OpenCode 1.19.5 and Pi 1.4.0, five turns each, on Flash-Next Bare Speed, 2.12.2 and the release code one after the other (October 30) The OpenCode 1.19.5 and Pi 1.4.0 rows above, with no failed request on either version
The same two clients on Flash-Next Optimized Speed with the release code OpenCode: 96.7% of prompt tokens from the cache, 20 of 20 follow-ups. Pi: 94.8%, 17 of 17. No failed request
OpenCode 2.2.5 on Flash-Next Optimized Speed, five turns of an agentic coding session at xhigh, with the SSD cache on (November 4) 147 requests, none refused or failed. 99.3% to 100% of each turn's prompt tokens came from the cache, up to a 144,623-token prompt. Each later turn's first token took 0.40 to 0.86 s
Pi 1.5.0 on Flash-Next Optimized Speed, five turns at medium (November 4) 34 requests, none refused or failed. 97.0% to 99.8% of each turn's prompt tokens came from the cache. Each later turn's first token took 0.20 to 0.30 s
OpenCode Desktop 2.2.5 driven through its window: a game, two questions, then /compact and an edit after it No HTTP 500, disconnect or retry. The session kept its title. The game it wrote ran, and the pause it added after the compaction worked
A screenshot in the middle of a Flash-Next conversation The image turn resumed from the previous turn's cache (435 of 435 tokens), and every verify step of the three turns ran on the compiled path
A long answer through the app's own engine: a game written at xhigh 47,068 tokens at 55.3 tok/s, temperature 1.0, a complete page
The small Flash-Next test model, four conversation shapes 2.12.2: 0 of 16 follow-ups reused the cache with the SSD cache on, and 0 of 12 with it off. 2.12.2 with its copying store (MTPLX_ONE_COPY=0): 16 of 16. The release code: 16 of 16 on, 16 of 16 off, and 12 of 12 with a 17,000-token first message. With the SSD cache off, plain chat follow-ups now resume at 3,740, 3,774 and 3,806 tokens, each previous prompt's end, where before #291's fix they resumed at 3,708 every turn
Tests for the fixes 36 new test files since 2.12.2, 4 new Swift test files, and new cases in existing ones. Each engine and server fix has tests that fail on 2.12.2 and pass here. The gauge fix was measured instead (see The app)
Flash-Next Optimized Speed, 2.12.2 and 2.12.3 taking turns (four boots, each started at the same chip temperature), a 4,096-token answer at xhigh (November 4) The same 4,096-token answer, byte for byte, at the native sampler with a fixed seed. Decode 69.9 tok/s on 2.12.2 and 69.8 on 2.12.3. Peak memory 86.1 to 86.9 GB on both. Sent again, 2.12.3 reused 77 of its 82 prompt tokens and answered in 0.15 s, where 2.12.2 read the prompt again and took 0.33 s
Codex CLI 0.167.0 on Flash-Next Optimized Speed, an 87,354-token prompt read from scratch, with Codex's idle timer set to 10 seconds and its retries off (November 6) Without #680, Codex ended the turn after 10 seconds ("idle timeout waiting for SSE"). With #680, the turn completed in 64 seconds, with the first token after 60.4 seconds. In a second request of the same size, MTPLX sent a status event every 5.0 seconds until the first token
Codex CLI 0.167.0 at its default settings, two coding turns on Flash-Next Optimized Speed (November 6) Both turns completed (198 and 76 seconds, 14 shell commands). The 12 tests Codex wrote pass. 95% and 96% of each turn's prompt tokens came from the cache
Test suites on the release code (November 6) pytest 11,643 passed, Swift 1,073 passed (1,075 run, 2 skipped), M1 to M4 rehearsal 763 passed, 0 failures

Known issues

  • Flash-Next on Ultras can still reach Metal's buffer limit after long uptime (#640). The engine now recovers by releasing idle conversations, but what holds the buffers on those Macs is not found yet. On single-die Macs (M3, M4, M5 Pro and Max), each time the compiled verify step is rebuilt with the hyper-connection read, about 9 Metal buffers and about 380 graph nodes stay behind inside MLX (#546). A rebuild happens about once per 8,000 tokens of growth past 16,000 tokens, and on every turn below that. A restart returns them.

  • OpenCode re-reads part of its prompt at each new turn when your home folder holds the same skill twice. OpenCode lists the skills it finds under ~/.claude/skills and ~/.agents/skills in its system prompt, and each new opencode run picks between duplicate copies differently, so the prompt changes a few thousand tokens in. Keeping one copy of each skill keeps the prompt stable. Tool loops inside a turn are not affected.

  • Flash-Next with plain decoding and its draft head still loaded reads the whole conversation again on every turn. That is --generation-mode ar or --no-mtp without --no-load-mtp, or the app set to plain decoding with the draft head left on. Add --no-load-mtp, which also frees the draft head's memory. 2.12.2 behaved the same.

  • Kilo loses track of tasks after a compaction (#653). It has the same shape as #635 and may be fixed by it, but it was not reproduced.

  • On models other than Flash-Next, with the SSD cache off, a plain API chat resumes from its first turn's prompt instead of the previous turn's. The re-read grows by one reply per turn, as in 2.12.2. Saving that prompt would copy the conversation's cache on those models. Keeping the SSD cache on (the default) avoids it.

  • Structured output needs the server extra. Homebrew and plain pip install mtplx do not install llguidance. The refusal now gives the command.

  • The known issues of 2.12.2 still apply. See the 2.12.2 release notes.

Upgrading

Run brew upgrade mtplx or pip install -U mtplx, or use the app's update check. You don't need to change any settings. The first turn of each Flash-Next conversation saved by an earlier version reads its prompt once.

Contributors

Full changelog: v2.12.2...v2.12.3

Contributors

rrosson, danhixon, and 38 other contributors
Assets 6
Loading