TensorFold 1.2.9

TensorFold 1.2.9

1.2.9 brings faster decode and long prompts, an Ollama-compatible API, and fixes for a wrong reply and a hang. Every
speedup below keeps replies unchanged.

Speed

• Nemotron on Metal (without an MTP head): concurrent streams share one round. On an M5 Ultra, two streams decode
  about 20% faster, four about 75%, eight 2.3 to 2.5 times and 24 streams 3.2 to 3.5 times (#3005).
• GLM-5.3-Flash on CUDA:
  • Long prompts' first token comes 0.6% to 13.7% sooner at TP=3 from 131,072 to 2,000,000 tokens, and 6.7% sooner at
    TP=2 at 1,000,000; shorter prompts at TP=2 stay within about 2% either way (#3159).
  • After a 1,040,384-token prompt, decode rounds are 5.05%, 9.36% and 11.36% shorter at 1, 4 and 8 streams (#2722).
• Qwen3.8-Flash-Next on HIP (gfx1151): one stream decodes about 6% faster, four streams 2.8% to 4.6% (#2697).
• Fewer CUDA launches for Flash Next draws (#2927), Nemotron 3.5 draws (15 a level instead of about 112, #3124) and
  Qwen3.8-27B prompts (#2925).

Models and features

• An Ollama-compatible API: /api/chat, /api/generate, /api/embed, /api/tags, /api/show, /api/ps and
  /api/version on the same engine paths as the OpenAI routes (#3342).
• tensorfold serve --config FILE.toml reads serve flags from a file, so a launch can be checked in (#2885).
• Linux: an opt-in memory-pressure governor driven by PSI, TENSORFOLD_MEMORY_PRESSURE=1 (#2156).
• Groundwork, not used by any served model yet: a CUDA image path for the dense engine (#2768), dots.mocr's vision
  kernels (#2767), and HIP's TCP fabric for two hosts without RDMA (#2961).
• An experimental Living Weights v2 controller in tools/living_weights_v2 (#3320).

Server and API

• /v1/models lists each model's input_types. A video sent to a model without video is a 400, not a 500 (#3019).
• Logprobs work with tools, stop strings and reasoning, and on the Responses API (#3206).
• With thinking off, a request's reasoning effort reaches the template (#3017).
• Replies say when they kept a tool call as text, in tensorfold.kept_tool_calls and two /metrics counters (#2814).
• Streams can carry the committed token IDs, opt-in (#3285).

Fixes

• Nemotron Omni on CUDA: a failed image request no longer leaks into the next reply (#2985).
• GLM-5.3-Flash at TP=3: prompt_logprobs over about 100 tokens no longer hangs (#1571).
• Living Weights: the drift gate no longer refuses a lesson over the writer's own wording (#2929).
• CUDA: the release server refuses a GPU it has no kernels for, by name, at startup (#2914).

For developers

• zig build test-host gathers the Linux host checks (#2095); flaky tests fixed (#2963, #3015, #3310, #3347, #3349,
  #3355); docs for TF_GLM_PROF (#2990) and a Jammed XSFP4 NZ5 row (#2981).

What changes by default

• GLM on CUDA: the TF_GLM_INDEX_CUDA kernels and TF_GLM_DSEL are on by default.
• Flash Next on CUDA: sampled draws are batched by default; TF_FLASHNEXT_BATCH_DRAWS=0 restores per-stream draws.
• Flash Next on HIP: on one RDNA 3.5 GPU (gfx1150 to gfx1153) serving alone, MTP drafts stop under a 0.8
  confidence floor; other HIP GPUs and families keep 0.3, and --mtp-confidence still overrides it.
• Server: --reasoning-effort no longer changes a thinking-off prompt.
• Server: a video_url part sent to a model without video gets HTTP 400 (invalid_request_error), not a 500.
• CUDA: the release server refuses, by name at startup, a GPU it has no kernels for.

The first three keep replies unchanged. The fourth changes the prompt of a thinking-off request when the server sets
--reasoning-effort. The last two turn a failure into a named refusal.

Thanks

@Bizuayeu, @Defilan, @garvrao80, @gilby, @jimthedj65, @jishnuvenugopal, @laudney, Mia's AI Lab (@MiaAI-Lab),
@Nipale-ai, @Shoestring-dev, @ss-cong, @tarsnake and @WebReflection.