CIRU Ling IU4 runtime v1.1.1

Ling IU4 Runtime v1.1.1

This patch hardens the ROCr idle-CPU correction shipped in v1.0 and retained
in v1.1.0. The corrected library itself is unchanged; the activation path is
now enforced inside the private runtime environment as well as by the CIRU
launcher.

Why this patch exists

The ROCm 7.15 wheel-bundled ROCr library can leave one EngineCore CPU thread at
100% while the server has no requests. The supported launcher already used a
checksum-verified LD_PRELOAD, but a direct vllm serve command could bypass
that environment and silently load the wheel copy.

v1.1.1 closes that gap:

• install_rocr_idle_fix.sh activates the corrected runtime inside the selected
  venv, preserves the original wheel file for recovery, and installs a Python
  startup bootstrap that enforces LD_PRELOAD before ROCm packages import.
• run_ling_ciru_iu4.sh still verifies and preloads the corrected runtime and
  now prints an explicit activation banner.
• verify_rocr_idle_fix.sh checks the library actually mapped by EngineCore and
  samples instantaneous thread CPU after startup.
• Existing non-default environments are supported with VENV=/path/to/venv.

Upgrade

Stop the running server, update the checkout, activate the fix in the same venv,
and restart:

git fetch --tags
git checkout ling-iu4-v1.1.1
VENV="${VIRTUAL_ENV:-$PWD/.venv}" \
  bash tools/ling_iu4/install_rocr_idle_fix.sh "$PWD"
MODEL_PATH=/path/to/Ling-3.0-Flash-CIRU-IU4 \
  bash tools/ling_iu4/run_ling_ciru_iu4.sh

With the server ready and idle:

bash tools/ling_iu4/verify_rocr_idle_fix.sh

The verifier reports PASS only when EngineCore maps a library whose checksum
matches the corrected local build. It also reports aggregate and hottest-thread
CPU over a fresh sample, avoiding the lifetime-average %CPU value shown by
some ps commands after a compilation-heavy startup.

Compatibility

Model weights, dynamic scheduler behavior, IU4/W4 kernels, MTP K1, context
support, and serving defaults are unchanged from v1.1.0.

Ling IU4 Runtime v1.1.0

Ling IU4 Runtime v1.1.0

This release adds a Ling-specific dynamic prefill budget to the public CIRU gfx1151 runtime. Model weights and the proven IU4/W4/MTP execution paths are unchanged.

What changed

• One runnable request uses the faster 8K prefill shape.
• A request that begins above 8K uncached tokens remains latched to 8K until prefill completes.
• Two or more waiting short/medium prefills can use the 16K ceiling.
• The scheduler returns to 8K immediately when concurrency drains to one.
• Existing decode priority is preserved.
• Static serving remains available with VLLM_LING_DYNAMIC_BATCHED_TOKENS=0.

Measured impact on one 128 GB Strix Halo APU

Against the v1.0 static-16K launcher, the order-balanced warmed C1/32K comparison measured:

• Prefill: +31.17%
• TTFT: −23.76%
• Wall time: −22.15%
• Decode: −0.17% (parity)

At C6/8K, the dynamic policy retained the static-16K serving path:

• Prefill: −0.59%
• Median per-request decode: −0.25%
• Wall time: +0.58%

The four order-balanced matched server loads were:

Position	C1 32K PP	C1 TG	C1 TTFT	C1 wall	C6 8K PP	C6 median TG	C6 E2E output/s	C6 wall	C6 accept
A1 static 16K	424.369	22.727	77.216	82.805	612.935	13.232	8.585	89.460	96.401%
C1 dynamic	558.645	22.707	58.656	64.250	608.033	12.828	8.488	90.486	95.652%
C2 dynamic	555.673	22.731	58.970	64.558	611.529	13.255	8.565	89.670	95.652%
A2 static 16K	425.172	22.786	77.070	82.645	613.897	12.916	8.566	89.653	95.652%

An earlier dedicated-profile oracle check is retained for completeness. It
asked whether one dynamic profile could exactly match separately tuned static
8K C1 and static 16K C6 profiles; it is not the deployed-default comparison:

Boundary	Profile	PP	Median TG	E2E output/s	TTFT s	Wall s	Accept
C1 16K	Static 8K oracle	645.290	22.430	4.120	25.390	31.052	96.92%
C1 16K	Dynamic	632.705	22.610	4.060	25.895	31.513	96.92%
C1 32K	Static 8K oracle	567.140	22.760	2.020	57.780	63.359	100.00%
C1 32K	Dynamic	555.525	22.797	1.983	58.986	64.557	100.00%
C6 4K	Static 16K oracle	638.240	14.880	16.290	38.510	47.157	96.91%
C6 4K	Dynamic	630.710	14.260	15.980	38.970	48.062	97.16%
C6 8K	Static 16K oracle	601.760	11.310	8.460	79.050	90.832	96.40%
C6 8K	Dynamic	599.920	12.920	8.410	81.360	91.273	96.65%

The dedicated static-8K oracle was 1.95% and 2.05% faster in C1 wall time at
16K and 32K. Dynamic C6 wall was +1.92% at 4K and +0.49% at 8K versus the
dedicated static-16K oracle. Those small gaps do not justify another serving
profile; the order-balanced comparison above is the causal release comparison.

Dynamic C1 context sweep

One excluded 512-token warm request was followed by one uncached measured pass
at every requested context, with exactly 128 forced output tokens:

Input	PP tok/s	TG tok/s	Wall s	MTP acceptance
2K	720.688	24.688	7.990	98.44%
4K	438.732	23.828	14.704	96.92%
8K	675.071	24.360	17.356	95.38%
16K	519.984	22.725	37.136	98.44%
32K	559.334	21.819	64.449	95.38%
64K	437.604	18.568	156.650	92.42%
128K	342.860	15.963	390.310	96.92%

These are intentionally one-pass boundary measurements, not best-of results.
The isolated 4K row did not reproduce after a same-concurrency warm batch: the
warmed C1/4K row below measured 700.276 PP tok/s. Both values are retained so
the anomalous one-pass result is visible rather than silently discarded.

Dynamic 4K concurrency ladder

Each row used one excluded same-concurrency warm batch followed by one measured
batch of unique uncached 4096-token prompts and exactly 128 output tokens per
request:

C	Aggregate PP	Median/request TG	Aggregate TG	E2E output/s	Wall s	MTP acceptance
1	700.276	23.578	23.578	11.392	11.236	96.92%
2	704.722	19.851	39.458	14.173	18.062	97.67%
3	688.311	17.704	53.107	15.343	25.027	96.91%
4	671.804	16.416	65.186	15.910	32.181	97.29%
5	661.500	15.088	74.887	16.228	39.439	96.60%
6	651.611	14.202	84.525	16.435	46.730	96.15%
7	641.725	13.345	92.634	16.508	54.276	96.48%
8	642.692	12.743	100.015	16.804	60.937	95.03%
9	645.142	9.140	80.513	17.025	67.665	95.24%
10	647.727	7.060	68.972	17.168	74.558	95.40%

C8 is peak aggregate decode. C9 crosses from two to three 16K prefill waves.
The launcher remains at six default slots because C8 improves end-to-end
output by only 2.25% over C6 while increasing batch wall time by 30.4%.

Full-agent dynamic-build validation

The thinking-enabled HermesAgent-20 C6 run measured:

Metric	Result
Score	90
Outcomes	16 pass / 1 partial / 3 fail
Wall time	1,387.878 s (23:07.9)
Prefill	279.977 tok/s
Median/request decode	4.817 tok/s
Active aggregate decode	18.413 tok/s
Suite output / wall	17.337 tok/s
End-to-end throughput	52.904 tok/s
Mean TTFT	11.259 s
MTP acceptance	90.989%
Prompt / computed / cached tokens	484,645 / 355,774 / 126,720
Generated tokens / requests	24,062 / 121

This is a standalone dynamic-build validation, not a matched comparison to an
older run with different timeout or sampling conditions.

Validation

• HumanEval/0–5: 24/24 passing natural completions across four independent launches.
• Full HermesAgent-20 thinking-on C6: score 90, 16 pass / 1 partial / 3 fail.
• All 62 requests in the context and concurrency sweeps had exact completion
  counts, zero preemptions, zero prefix reuse, and no runtime faults.
• Zero preemptions and zero GPU/runtime faults in the retained matched scheduler
  and full-agent runs; HermesAgent-20 had no scenario timeout.
• The ROCr idle-CPU correction from the previous public build remains included.

Upgrade

git fetch ciru ciru/ling3-gfx1151-iu4 --tags
git checkout ling-iu4-v1.1.0

Restart the server with tools/ling_iu4/run_ling_ciru_iu4.sh. Existing model files and the built IU4 extension do not need to be replaced.