Repository navigation
First Series X run: xllama 1.7.0.1495, LFM2.5-350M and 1.2B, CPU road #434
Replies: 2 comments 2 replies
|
Thank you, Joshua, this is the first Series X report we've had and it's really helpful. I especially appreciate that you kept the on-screen and LAN end-to-end timings separate. The jump after marking the package as a Game is very interesting; I'd love to understand it better. Yes, please open a PR with the raw runs under |
|
All three untested Series X paths from my earlier note are now measured on this console. Xbox Series X, xllama 1.7.0.1626, Game designation, app memory limit 5120. GPU path, GGUF D3D12. LFM2.5-1.2B-Instruct, 10 timed runs, 99.7 tokens per second end to end, against 37.8 on CPU, about 2.6 times, above the Series S figure. App log shows all 17 layers offloaded. One honest note, at temperature zero the CPU and GPU outputs were not token identical, 121 versus 125 tokens, both clean. Image generation, SD-Turbo, works. The LAN API will not provision the model on this build, the app does it on first generate, about 2.52 gigabytes, seven files verified. Through the API, 512 by 512 takes about 8 seconds at one step and 9.4 at four steps, seed 42, while the app's own pipeline is faster because it does not reload each time. Same seed outputs matched within each set, peak memory around 2920 megabytes, no crashes. Training works. 21 usable samples, 20 fixed samples plus the app's one. The catalogue had ONNX models, so we supplied a SmolLM2 360M Instruct GGUF, 725 megabytes, hash verified. Eight epochs, 586 seconds, final loss 0.484. It merged a personalized model into the app's storage, and that model is now listed and active. Per run timings, hashes, and log excerpts are kept and I can share any on request. Joshua Henne |
Uh oh!
There was an error while loading. Please reload this page.
Field report from Eidon Labs Inc. We ran xllama 1.7.0.1495 on an Xbox Series X in Developer Mode (console OS 10.0.26100.9662) and measured text generation two ways, kept separate throughout: (a) the app's own on-screen internal timing, and (b) LAN API end to end timing from a second machine on the local network (wall clock around each call, prompt processing and network included). The two measures are never blended.
App internal: about 103 tok/s on LFM2.5-350M, one short run of 31 tokens, read off the console screen.
LAN API end to end, fixed prompt of 31 tokens, batches of three runs unless noted:
LFM2.5-350M, before the app was marked as a Game package: batch averages about 35.4 to 38.8 tok/s.
The Founder then marked xllama as a Game in Dev Home. After that change, on the same console, app, model and road: short batch averages about 81.8 to 90.1 tok/s, and ten full 1024-token generations held about 95.7 tok/s on average (best 96.0). The timing lines up exactly with the designation change; we state this as a strong correlation, not a proven cause.
LFM2.5-1.2B-Instruct (catalogue file LFM2.5-1.2B-Instruct-QAD-Q4_0.gguf, 695,755,488 bytes, LiquidAI, LFM Open License v1.0), CPU road forced via the app's gguf_gpu_layers override: three runs at about 37.2, 37.3 and 36.8 tok/s, average about 37.1 tok/s end to end.
Full paper with the complete raw record, method, replication packet and hashes:
https://muse.ai/s/project-scorpio-first-inference-xki6pdxy7xgxjuw/wh9yeiC-VcH6LMkLaY_ZJrWTZKOCTUPSspuX9wR0RJM
If the data is useful for the project's benchmark table, we are glad to contribute the raw runs in the repository's CSV convention under bench/results/. Please credit Joshua Henne / Eidon Labs Inc. wherever this data is reused. Thank you for building xllama; the readme's note that Series X testing was the next step is what sent us to the console.
Joshua Henne CEO Founder
Eidon Labs Inc.
Contact: Joshuahenne1010@outlook.com
Cell: 1-447-288-7635
All reactions