TensorFold 1.0.2

GLM-5.3-Flash on two M5 Ultras:

Several requests at once. Eight concurrent requests decode at 199 tokens a second in total, against 100 in 1.0.1, and each stream equals its solo run.
Both Macs now split the attention heads, the dense layers and the vocab head as well as the experts. One stream decodes plain at 91 tokens a second, from 83. With drafts it decodes 114 to 131 tokens a second on prose and 138 to 148 on code, each reply equal to its plain one. A fresh 7,300-token prompt reaches its first token in 4.4 seconds instead of 6.5.
Prompt caching. A resumed prompt equals a fresh one token for token, and a 7,300-token conversation resumes in 0.18 seconds. A new conversation that starts with the same system prompt and tools resumes that whole prefix: its first token came 0.06 seconds after a 4,700-token system prompt.
--learn keeps those shared prefixes on disk, so a restarted server resumes them too: 0.64 seconds to the first token instead of 4.43. An upgrade keeps them when a startup check finds its prompt passes give the same bits. --learn-gib caps their disk, 32 GiB by default. Start both Macs with --learn.

Homebrew: brew upgrade tensorfold.

Navigation Menu