Skip to content

Loom v0.2.0

Latest

Choose a tag to compare

@github-actions github-actions released this 16 Nov 22:39
· 70 commits to main since this release
Immutable release. Only release title and notes can be modified.
0cd370f

Loom 0.2.0 expands complete-program C++ authoring, runtime-provided managed objects, stateful finite services, and packaged AMD GPU/NPU and Vulkan execution. It expands the shared target compiler across x86, AIE2P, AMDGPU, SPIR-V, and WebAssembly while removing fixed and superlinear costs from JIT compilation, graph construction and updates, GPU launch and replay, pipeline planning, and long-context serving.

Highlights

  • Editable C++ programs: Author and round-trip function libraries, command graphs, finite pipelines, inherited objects, structured control flow, range loops, initializer lists, standard math, and target-specific Low assembly through C++.
  • Managed native objects: Declare opaque provider-owned types in Loom IR and preserve owning and borrowed handles through linking, VM calls, control flow, spills, and final-use retirement.
  • Client-compatible finite services: Drive finite work through source-defined batched epochs, retained host/device state, partial result publication, and in-process completion; ordinary chat clients can preserve extension metadata and nullable defaults through shared source policy.
  • Packaged device execution: AMD GPU execution is consolidated on the libamdf-backed HAL under the existing amdgpu, rdna, and cdna device names; the SDK adds amdf-info on Linux and Windows, and installed distributions can execute through a system Vulkan 1.3 driver.
  • Native AMD GPU launch and replay: PM4 and AQL queues preserve exact dependency, cache, completion, argument, and publication facts through packet emission, removing redundant packets, posting reads, repair shaders, instruction invalidations, size-dependent preparation cliffs, and per-dispatch completion overhead.
  • Elastic long-context memory: Text caches grow from shared physical backing, compact contiguous mapping updates, and preserve cached conversations through pressure that cannot actually reclaim space.
  • Large graph management: Shared linear analysis and topology-owned indexes make 4,096-node graph construction, cloning, scheduling, replacement, and empty-node elision practical at microsecond scale without adding launch work.
  • Target and JIT efficiency: Native AIE2P spilling, resident and replaceable XDNA phase code, native x86 managed-object calls, broad cross-target math, reusable command preparation, and sparse pipeline planning extend useful coverage while bounding compiler work.

Author complete Loom programs in C++

Loom C++ is now an editable representation for more than isolated kernels. Function libraries preserve public signatures and private helper closure; command programs preserve specialization and launch bindings; and finite pipelines preserve placement, channels, ownership, pooled storage, asynchronous transport, and target selection. A mixed source package can combine C++ and authored Loom providers and enter the ordinary linker and JIT.

For example, a command schedule remains structured in both C++ and Loom IR:

[[loom::command_program]]
void decode(unsigned tokens, [[loom::launch]] Parameters parameters, Output output) {
  [[loom::serial]] {
    loom::command::launch<decode_layer>(tokens, parameters, output);
    [[loom::concurrent]] {
      loom::command::launch<prefetch_left>(parameters, output);
      loom::command::launch<prefetch_right>(parameters, output);
    }
  }
}

Ordinary and generic lambdas, forwarding-reference helpers, structured bindings, member aliases, aggregate constants, and overlapping mutable references now preserve their C++ identity where required while remaining allocation-free SSA values where identity is not observable. A compile-time traversal can therefore carry a capturing generic lambda through recursive helpers without introducing a lambda-specific ABI:

template <int Index, class Function>
void apply(Function&& function) {
  function(Index);
  if constexpr (Index < 2) apply<Index + 1>(function);
}

int accumulate(const int* input, int index) {
  int sum = 0;
  apply<0>([&](auto value) { sum += value * input[index]; });
  return sum;
}

The source boundary now also preserves nonvirtual base classes, copy and move operations, temporaries and destruction, deferred semantic-value construction through branches and loops, early returns and switches, range-based loops, and std::initializer_list backing arrays. Fixed-array accessors inside value records and closures can return a nonescaping pointer to the array's current SSA value even when the record also contains semantic pointer fields; later writes through the owner or borrow remain visible without allocating byte storage. Ordinary <cmath> calls map to typed Loom scalar operations instead of host libc calls, so source such as std::sqrt(value) + std::fabs(value) becomes target-independent scalar.sqrtf, scalar.absf, and scalar.addf operations. Explicit member-template instantiations check their selected bodies even when no call otherwise reaches them, while extern template remains declaration-only. These features share the same object, lifetime, and structured-control representation rather than adding target-specific frontend paths.

Foreign frontends can now use the dependency-free loom.importers package shipped in the base wheel to construct, verify, print, and serialize canonical Loom modules. Parser and source-language dependencies remain with the integrating project; the SDK owns the typed target-side builder and bytecode boundary.

The production pipeline witness translates a complete 32-core array program into 1,011 lines of C++, preserves its memory pools, 186 channels, 52 strands, and three final native programs, and produces the same qualified embedding results through source and precompiled paths. An executed C++ XDNA pipeline round-trips to the same 6,400-byte artifact and external-traffic shape as its authored Loom form.

See function-library export, complete linked pipeline projection, lambda closures and forwarding references, nonvirtual inheritance, structured C++ control-flow round trips, range-based loops, standard-library math, explicit member-template instantiation, array accessor borrows, and the Python importer boundary.

Carry runtime-owned objects through Loom programs

Loom IR can now name opaque types supplied by a native runtime provider without exposing their layout to the compiler. ref<T> is an owning handle whose transfer must be balanced on every normal path; ptr<T> is borrowed and preserves the caller's ownership contract. Both survive ordinary calls, branches, linking, bytecode, VM register allocation, spills, and overflow call packets with exact ownership modes.

For example, a provider can return one owner and one protected borrowed handle. Consuming the explicit owner does not invalidate the saved pointer, and its protection retires after the final use:

type.decl import("images", "frame") @Frame

func.decl import("images", "capture") @capture() -> (ref<@Frame>, ptr<@Frame>)
func.decl import("images", "consume") @consume(%frame: ref<@Frame>)
func.decl import("images", "width") @width(%frame: ptr<@Frame>) -> (i32)

func.def public @capture_width() -> (i32) {
  %owner, %saved = func.call @capture() : () -> (ref<@Frame>, ptr<@Frame>)
  func.call @consume(%owner) : (ref<@Frame>)
  %width = func.call @width(%saved) : (ptr<@Frame>) -> (i32)
  func.return %width : i32
}

ref.duplicate creates an independent owner when two consumers need the same object and preserves borrowed mode when duplicating a pointer. Public artifact emission rejects unconsumed owners or missing lifetime preparation, including direct Low and custom-pipeline paths. Programs without managed types skip function-level lifetime analysis after the canonical type-table check.

Native x86 object functions can carry owners, borrows, and independently sized counted-reference views through an explicit status-returning convention into C. Borrowed collections cross that boundary without per-element retains, owning results publish only after success, and a failed native operation releases local owners before returning its original status. Ordinary SysV functions keep their platform convention. The shipping compiler implements this transport for authored Loom IR; the typed C++ facade and a public logical-import binding or native program loader remain outside v0.2.

See runtime-provided object types, source handle duplication, borrow-preserving VM transport, and native x86 object and reference transport.

Acquire and embed stateful finite model services

loom-fetch accepts Hugging Face and GitHub repository URLs or explicit provider references, resolves one immutable repository revision, and selects models and presets without requiring users to construct raw model.json URLs:

loom-fetch huggingface:OWNER/REPOSITORY:PRESET@REVISION
loom-fetch https://github.com/OWNER/REPOSITORY/tree/REVISION
loom-serve --device=amdgpu --model=./installed-model

Naming an installation directory selects its model.json; explicit model-card and deployment-document paths remain accepted.

loom-serve chat requests can now arrive in the shape ordinary clients emit. The native parser accepts extension metadata across messages, supported content parts, media descriptors, tool-call history, and streaming options while preserving strict validation of required fields and content kinds and forwarding the original request to source policy. Explicit null values for optional native controls use omission defaults, and a non-null max_completion_tokens takes precedence over max_tokens. Accepted extension fields remain source-visible no-ops unless source policy consumes them. Shared Qwen policy applies the same rule to sampling, reasoning, and template controls, so serialized nulls preserve deployment defaults instead of rejecting the request. Integrations no longer need a request-sanitizing proxy solely to strip metadata or nullable defaults.

Finite execution no longer belongs to the HTTP server or to a one-shot batch. An application can drive the shared runtime directly, and a model's Loom control program can retain per-request state across several compatible device epochs:

prepare_request(state, input) -> (request_state, units, compatibility_tag)
prepare_batch(state, stage_tags, batch: ptr<finite.batch>)
run(state, stage, input, output, batch: ptr<finite.batch>)
collect_batch(state, batch: ptr<finite.batch>)

The service owns admission, cancellation, accepted queue captures, and final retirement. Source code owns work offers, compatibility classes, persistent device ranges, successful progress commit, and ordered partial results. Short requests can join an ongoing computation and complete before longer peers; a logical result can span several numerical cohorts while earlier rows retain their original storage. HTTP remains one frontend over the same lifecycle, and device consumers can retain results together with their producer dependency for downstream composition. This release proves that boundary for finite embedding, decision, and image-to-text projection paths; extending it across the remaining service families remains active work.

Image programs now produce row-major RGBA8 directly instead of returning planar F32 RGB for host conversion. At 1024×1024, the final device allocation and readback each fall from 12 MiB to 4 MiB. PNG output now preserves an explicit alpha byte, so model packages must migrate their final store together with the service contract.

Serving work in this release also removes several avoidable latency and memory costs. These representative qualified results retain their original workload boundaries:

Production witness v0.1-era path v0.2 path
Late 24-token prompt beside a 32K prompt and eight retained decoders, time to first token 1.88 s 0.61 s
Preparation of a 1,039,569-token input while two decoders remain active 22.43 s 674 ms
Recording 10,928 model-preparation commands with 64 existing workers 66.51 s 48.60 s

Long-context caches now derive addressability from the allocation owner and draw physical pages from one shared pool. In the qualified million-token configuration, removing the context-sized session token array saves 512 MiB across 128 empty sessions. Representative first-page backing that previously required 2.1-2.6 GiB per source configuration now shares one 64 MiB parent. Optional memory_bytes tightens the pool-wide budget; it does not create a second capacity ledger.

Mapping preparation now preserves page ownership while sending contiguous physical ranges to HAL. In a complete 32K-context serving comparison with three retained forks and a 16K-token replay, median peak process RSS fell from 514 MiB to 309 MiB. Descriptors entering HAL fell from 563,905 to 509 while native mapping updates remained 509; long-context and replay medians changed by less than 0.2%. This is a host-memory result for that workload, not an inference-throughput claim.

Native embedding hosts now establish one CPU eligibility budget at construction and carry it through Task topology and asynchronous I/O worker creation. A process restricted by taskset, Slurm, or another launcher no longer has a node-local I/O worker expand outside that budget. This changes native host, thread-affinity, topology, and async-runner call signatures; the migration table gives the required caller updates.

See repository model references, mixed-request responsiveness, shared physical text caches, embeddable finite services, batched source continuations, retained finite device state and results, model-directory loading, compact elastic mapping updates, client-compatible chat requests, host CPU budget preservation, and nullable shared Qwen defaults.

Use packaged GPU and NPU execution

Loom tools, examples, serving, sanitizers, and hardware qualification now use one libamdf-backed AMD HAL through the existing --device=amdgpu spelling. The native path supports RDNA and CDNA execution, shared CPU/GPU/NPU memory contracts, explicit compute-resource selections, cooperative execution, virtual mappings, and reusable command buffers. The former ROCr-backed driver, its legacy-amdgpu selector, and its separate runtime and memory ownership paths are removed.

The SDK installs amdf-info, and the release includes standalone Linux and Windows archives for machine inventory and device-health checks without ROCr, XRT, Python, or a repository checkout:

amdf-info --format=json
amdf-info --device=rdna --activate
amdf-info --device=xdna --activate --format=json

The installed SDK also includes the Vulkan HAL provider. It dynamically loads the host's Vulkan loader, requires Vulkan 1.3 with buffer device address support, and does not bundle a loader or vendor driver. Machines without Vulkan can still start the SDK. Adding this execution path increased the compressed Linux and Windows wheels by 1.23% and 1.09%, respectively, under the release toolchain comparison.

Reduce PM4 and AQL launch and replay work

The native AMD GPU stack now carries exact queue facts to the point that packets are published. Same-queue dependencies join existing execution and cache boundaries; one PM4 release can prove both native completion and host retirement; immutable instructions are published once during loading; and ordinary AQL launches place directional acquire, release, and completion on one dispatch packet. Kernel arguments, profiling samples, native commands, replay values, and CPU templates have separate memory policies and reuse boundaries instead of sharing one conservatively uncached arena.

Device-local AQL rings now publish their body and valid header with ordered stores instead of a native flush and posting read. Bounded fixed-argument recordings publish pre-encoded packets directly, while mutable recordings prepare on the CPU independently of argument size. Qualified queues keep owned kernargs on their local memory route and can share one physical completion across a bounded run while preserving each dispatch's dependencies, resources, and failure outcome. Large fixed PM4 recordings preserve invariant address repairs across reuse and can skip the repair shader. Queue captures grow from reclaimable slabs rather than one host allocation per operation.

Buffered streams now keep intermediate producer/consumer work at device scope and prepare ordinary AQL recordings in native queue-sized chunks. The former 32-dispatch, 8 KiB packet, and 256-fixup thresholds no longer switch to a resident repair path. Reusable fixed-address recordings publish argument bytes once, and warm PM4/AQL/SDMA recording reuses placement-specific native ranges until explicit trim. Recorded PM4 transfers now select among confirmed scalar packets, an exact-range resident blit shader, and bulk DMA from known size and alignment. Short tails use bounded 8/4/2/1-byte accesses instead of serial byte loops, while fixed copies and fills through 1 MiB can avoid DMA setup and dependent joins.

Representative qualified results retain their original timing boundaries:

Witness Earlier measured path v0.2 path Outcome
W7900 device-local AQL ring, 32-dispatch CPU submission 319.61 µs 27.38 µs 91% less submit time
MI300X device-local AQL ring, 32-dispatch CPU submission 125.47 µs 34.67 µs 72% less submit time
MI300X completed AQL launch with 392 argument bytes 152.98 µs 8.45 µs Removes the 392-byte preparation cliff
MI300X completed AQL empty-kernel chain, per kernel 5.011 µs 1.445 µs 71% less; HIP reference 2.303 µs
MI300X completed AQL chain with changing 336-byte arguments, per kernel 11.206 µs 2.837 µs 75% less; HIP reference 2.871 µs
MI300X completed 32-kernel AQL graph replay with 336-byte arguments 93.704 µs 91.962 µs HIP reference 98.165 µs
W7900 completed PM4 tiny launch 10.62 µs 3.941 µs Successive qualified steps; final HIP reference 3.889 µs
W7900 Q8 fused projection motif, GPU event interval 215.29 µs 28.940 µs Successive qualified steps; final HIP reference 29.482 µs
MI300X fixed-argument 64-dispatch graph completion 149.77 µs 106.86 µs The 4,096-dispatch resident path remains effectively unchanged
MI300X buffered 33-dispatch resident chain 393.6 µs 108.3 µs Removes the 32→33 preparation cliff; HIP reference 129.4 µs
gfx1100 flush-each 64-dispatch recording 6,133.0 µs 519.2 µs Warm native allocation reuse; HIP reference 326.3 µs
gfx1151 completed graph of 256 four-byte copies 869.1 µs 157.6 µs Scalar PM4 packets; HIP reference 512.4 µs
gfx1151 completed graph of 256 four-byte fills 856.2 µs 117.4 µs Scalar PM4 packets; HIP reference 533.9 µs
gfx1151 completed graph of 256 twelve-byte copies 488.62 µs 214.23 µs Exact-width shader tails; HIP reference 529.30 µs
gfx1151 completed graph of 256 128 KiB copies 2,039 µs 531 µs Resident bulk shader; HIP reference 926 µs
gfx1151 completed graph of 256 128 KiB fills 1,814 µs 367 µs Resident bulk shader; HIP reference 643 µs

The PM4 launch and Q8 rows summarize successive matched A/B landings over the same witnesses, not one direct endpoint comparison. The final AQL comparison uses identical precompiled workloads and fresh-process rotating runs on MI300X: ordinary sustained chains improve by 28–38% over the preceding path, changing-argument chains are within 1.4% of HIP, and the largest measured graph replay is within 1.6%. A combined callback-and-download observation remains 246 ns per kernel slower than HIP after a 64-kernel burst; these are launch and observation measurements rather than model-throughput estimates. The complete checked 1.7B Q8 inference witness finishes prefill at 8,762.26 tokens/s versus HIP's 8,933.84 and decode at 115.95 tokens/s versus HIP's 114.67, with bitwise-identical logits across the compared runtimes. These results establish decode parity and near-parity prefill for that workload; they are not a blanket claim for every kernel or device.

The ownership work also removes host churn that would otherwise dominate large recordings. In one complete serving trace, allocations nested in 377,963 queue reads fell from 378,288 to 6,495, and whole-trace allocations fell from 481,332 to 94,517. A reused large fixed PM4 recording with no changing repairs drops from 53 to 12 DWORDs of invocation packets before surrounding waits and retirement.

Resident shader transfers trade about 132 additional bytes of recorded storage per transfer and compute occupancy for lower latency. In a matched 128 KiB copy workload beside sustained compute, copy latency fell 44% relative to official HIP while neighboring compute mean latency rose 3.8%; these are resource-competition results rather than a universal crossover. Exact native dependencies now also remain reachable for every live reusable submission instead of falling back to host timeline retirement when a later producer supersedes the semaphore's newest value.

See Vulkan SDK execution, native Loom GPU execution, native memory and execution contracts, reclaimable queue-capture slabs, fixed-argument AQL replay, amdf-info, one-packet native AQL launches, retirement of the legacy ROCr driver, invariant PM4 replay repairs, device-local AQL publication, separate submission memory policies, shared PM4 completion, local PM4 dependency joins, load-time instruction publication, size-independent AQL argument preparation, local kernarg publication with amortized completion, device-local stream reuse and chunk-scaled AQL preparation, warm native recording storage, scalar PM4 transfers, size-adaptive recorded transfers, exact-width shader tails, reusable native dependency retention, and resident bulk transfers.

Scale graph construction, updates, and empty structure

Native HRX and HIP graphs now share one compact linear analyzer. Each graph owns its incoming topology once; sorting builds outgoing adjacency only when creation order is not already topological; recording consumes the analyzer's barrier decisions directly; and temporary scheduling storage returns to the compiler arena after instantiation. Construction, cloning, root enumeration, and whole-graph replacement reuse that topology instead of searching unrelated nodes and edges.

Empty and disabled nodes retain their identity and dependency semantics without becoming runtime partitions, semaphores, or barrier submissions. A graph of 4,095 empty nodes and one host callback therefore needs one execution partition instead of 4,095 internal semaphores. Public graph APIs, accepted topology, and launch ownership are unchanged.

Retained HIP graphs now store a 40-byte node header followed by only the active operation payload, dependencies, and argument capacity instead of reserving a 520-byte largest-kind union for every node. A 4,096-node empty description uses 256 KiB of arena backing instead of 2,176 KiB; a dispatch description with 16 argument bytes uses 1,216 KiB instead of 2,240 KiB; and a host-callback description uses 448 KiB instead of 2,176 KiB. The same savings apply to each template and executable snapshot, so retaining 64 dispatch templates and snapshots saves 128 MiB. Native HRX and HIP nodes also carry HAL-ready commands into one shared recorder rather than reconstructing them from frontend query metadata. Public graph APIs, stable handles, replay synchronization, and GPU program formats are unchanged.

Replacement preparation now allocates and records outside the executable launch lock, then publishes the completed recording under a short shared boundary. Installed graphs continue launching while allocation or driver work prepares the next version, and failed preparation preserves the installed state. Compiled HIP-on-HRX blocks also store only their active operation payload: a recorded command-buffer block shrinks from 136 to 24 bytes, and a real 4,096-callback graph reduces compiled-arena use from 640 KiB to 320 KiB while native HRX retains its existing compact layout. Both frontends consume the same allocation-free ordinary replay path.

Native HRX and HIP recording now also derive partition visibility through that shared path. Device-only results stay within device dependency scope across partitions, while every partition boundary—including one containing only dispatches—publishes the host visibility its public graph semantics require. This removes frontend-specific release flags without changing graph APIs or launch ordering.

CPU-side graph operation Before v0.2 path Speedup
Analyze a 4,096-node chain with added edges 60.27 ms 29.05 µs 2,075×
Construct a 4,096-node wide join from individual edges 6.58 ms 137 µs 48×
Clone a 4,096-node chain 6.83 ms 85.45 µs 80×
Accept a 4,096-node wide-join executable update 10.15 s 548 µs 18,545×
Instantiate a 4,096-node empty-chain control 270.8 µs 111.4 µs 2.4×

These are graph-management costs, not GPU execution or inference timings. Ordinary 4,096-node copy and dispatch controls remain close to their prior CPU cost, while the structural cases no longer become superlinear or manufacture execution work.

See compact linear graph scheduling, scalable graph construction and cloning, topology-scaled graph updates, empty-node execution elision, compact graph storage with shared recording, nonblocking graph replacement, compact compiled replay storage, and shared graph partition visibility.

Improve AIE2P, XDNA, x86, and shared JIT compilation

AIE2P FP16 pair dots now lower to one bounded packet program with two native VMACs instead of scalar expansion. The selected path emits 61 Low operations, stays within the 24-unit vector-register budget without spills, and replaces a scalar-expanded witness that reached a 368-unit scalar-register peak.

When live values do exceed the physical register banks, AIE2P can now spill them into ordinary tile SRAM. The compiler sizes and places worker and direct-leaf helper frames alongside authored buffers, channel rings, and constants, reuses frame backing across sequential calls, and diagnoses the exact tile capacity before publishing an image. Native transfers cover scalar, address, vector, accumulator, predicate, FIFO, and block-floating-point registers. This enables pressure-heavy wide integer arithmetic, shifts, loads, and conversions that previously failed allocation.

XDNA physical planning now compiles one exact resident protocol once and links it independently at every placement. A 16-placement stress array reduces native compiler entries from 16 to 2 and full artifact compilation from 539.4 µs to 125.8 µs (4.29x), with byte-identical artifacts. Repeated DMA bodies are also shared: an eight-site, four-descriptor witness drops from 2,630 bytes to 1,440 bytes, and the complete 32-core embedding program fits its 16 KiB per-tile instruction limit without spill traffic.

Pipelines whose combined native code exceeds a tile's 16 KiB instruction store can now retain their controller, loader, shared helpers, tensor storage, and channel state while replacing retired phase code within one invocation. A qualified 32-worker, 28-layer embedding program that previously exceeded instruction and catalog capacity executes as one dispatch with a largest live footprint of 16,372 of 16,384 bytes and a maximum per-memory-tile catalog of 52,748 of 80,384 bytes. Complete request timings improve 2–6% against the earlier qualified replacement build; the result establishes fit and output quality for that workload, not a general throughput claim.

When a complete phase fits beside the replaceable region, the compiler now keeps it in the initial image and removes its load, descriptor, and delivery route. In the qualified 32-worker workload, groups requiring replacement fall from 230 to 181 and complete 256- and 513-token requests improve by 2–4%, while compilation time and peak memory remain effectively flat. The executable grows by 22,984 bytes because fewer loaded callers share identical catalog payloads.

Retained FIFO panels can now hold more than 63 logical records when the selected replay protocol consumes one physical semaphore credit, and SRAM streams can broadcast vertically into a compute row before branching across columns. A representative 32-core projection retains 256 KiB per column and reads 2 MiB of activation input across eight columns instead of the 16 MiB required by rereading each of eight feature groups. These are logical external-transfer counts for that execution shape, not end-to-end latency or memory-controller measurements.

Resident array pipelines can also reuse multicast routes across SRAM senders proven to execute serially, forward private framed paths without packet arbitration, and preserve native-call memory-space effects across representation changes. The compiler can therefore retain data on the array and overlap DMA with nonconflicting native work instead of exhausting links or conservatively joining every transfer.

Those ownership contracts now survive composed resident child invocations. A complete 32-core INT8 projection with M=624, K=6144, and N=6144 executes as either a root or concurrent child while retaining partial sums on die and draining only the complete result. Root and child forms move the same 49,250,304 logical operand/result bytes and measure 1.915 ms and 1.965 ms, respectively, including ingress and output completion: about 2.6% for the child boundary without additional operand traffic.

XDNA native emission now chooses compact constant moves after values become concrete. Existing program comparisons save up to 128 bytes per tile without source annotations or another pass. Shared pipeline planning also scales with actual resource participation instead of multiplying every invocation by every physical engine: a synthetic 1,024-independent-branch case drops from 56.48 MiB to 983.4 KiB of planner arena and from 22.52 ms to 134.21 µs. Dense 256-phase planning pays about 5% more arena storage and 13 µs in the same directional benchmark; complete finite-input artifacts remain byte-identical.

Feature-qualified AVX-512 profiles now preserve both FP8 source formats through scalar and 128-, 256-, and 512-bit function carriers, matching the source-type support already present in their register and ABI mappings.

Explicit afn logarithms now lower on x86 without host libm calls. One target-neutral recipe covers logf, log2f, log10f, and log1pf across scalar and vector f64, f32, f16, bf16, and both FP8 formats; strict logarithms remain rejected until x86 has a contract satisfying their stronger numerical requirements.

Exact x86 ceil, floor, truncation, ties-to-even, and ties-away rounding now cover scalar and vector F64, F32, F16, BF16, and both FP8 formats. Approximate exp, exp2, and cancellation-safe expm1 cover the same formats across x86, AMDGPU, SPIR-V, and WebAssembly while retaining native F32 paths. Approximation permission remains explicit; ordinary strict operations are not silently relaxed.

Complete clustered floating sums on qualifying SPIR-V subgroup32 profiles now select direct XOR butterflies from the canonical reduction operation. One production source migration removes 77 SPIR-V instructions and 11.8% of the artifact, lowers compile-time median by 18.6%, and improves the isolated device mean by 1.3%. Incomplete or divergent clusters, partial subgroup64 workgroups, preserved arithmetic, and non-sum combining families retain their native reduction.

Shared fact propagation now converges complete retained-call scalar/vector distributions only after reference decomposition is stable, withdraws stale numeric identities through nested control flow, preserves constraints through comparisons and remainders, and materializes retained vector components in dependency order. These repairs prevent rejected or incorrect target lowering without adding target-specific recovery; the maintained unaffected control artifacts remain byte-identical.

Computed index arithmetic now selects its carrier from the complete signed range before target lowering. Valid negative coordinates remain representable, and AMDGPU and SPIR-V no longer truncate a wide product at native width before extending it. Single-use native-width consumers still narrow before carrier selection, vector legalization retains coordinate and slice bounds, and uniform wide AMDGPU arithmetic can remain in scalar registers. A uniform 4 * coordinate + 8589934592 kernel on gfx1100 falls from 19 to 17 instructions, 124 to 84 machine-code bytes, and eight to four VGPRs; gfx90a falls from 19 to 17 instructions, 108 to 84 bytes, and seven to four VGPRs. Native signed cases execute across x86, VM, AMDGPU, SPIR-V, WebAssembly, and XDNA; genuinely wide cases execute where the target has a wide carrier and receive precise diagnostics on native-only targets. The optimized compiler grows by about 5.7 KiB (0.026%), with unchanged representative workspace allocation and no material phase-time regression. VM artifact requests also diagnose unsupported kernel entrypoints instead of silently producing an empty program.

The public LoomC command path now creates one loomc_cmd_compiler_t for a frozen source index and reuses prepared native sources and staged selection plans across product builds. Its default 64 MiB retention budget is configurable and does not block oversized or contended preparations from compiling privately. Shared compiler call-graph construction for a 4,096-callee fan-out falls from 2.32 ms to 0.167 ms and uses 25.6% less graph-arena storage; native-only modules also skip semantic-call preparation they cannot consume. Generated launch counts also reuse one sampled SSA observation across compatible consumers: a complete recursive command reduces 126 generated count dispatches to three and independently published kernel requests from 165 to 42, without changing authored dispatches. Separately, an 18,538-value SPIR-V production compile eliminates all 11 oversized arena allocations, lowers requested system allocation by 2.03 MiB and peak system-live memory by 650 KiB, and emits byte-identical output.

Late compilation diagnostics now retain the selected export and source operation across AMDGPU, XDNA, x86, VM, WebAssembly, and SPIR-V emission. A multi-kernel private-storage rejection identifies the failing export in its text and exposes the same export_name through the structured LoomC diagnostic, including when bytecode retains locations without source text. Register-transfer, call-binding, and storage-overflow failures report at their owning operation instead of losing context behind a terminal status.

Across targets, generated descriptor, assembly, contract, emit-reference, and guard tables continue moving toward compact immutable representations with direct lookup and no runtime allocation. Five representative compaction changes remove about 274 KiB of installed libloomc data while preserving byte-identical emitted programs and effectively neutral JIT instruction counts.

See AIE2P FP16 pair dots, resident XDNA compilation sharing, retained cross-column SRAM reuse, x86 AFN logarithms, exact x86 rounding, reusable command preparation, cross-target exponentials, native AIE2P spilling, bounded compiler arenas, compact XDNA constants, sparse pipeline planning, resident route reuse and DMA overlap, bounded call-graph construction, composed resident XDNA ownership, shared generated launch counts, source-attributed compilation diagnostics, retained-call distribution convergence, AIE2P lock/stream progress, clustered floating-sum selection, retained vector component ordering, nested numeric-fact coherence, comparison and remainder constraints, signed index carrier selection, resident XDNA phase elision, and XDNA phase code replacement.

Behavioral changes and migrations

Affected surface Previous contract v0.2 contract Migration
C++ callable placement [[loom::force_inline]] and LOOM_FORCE_INLINE [[loom::inline]] and LOOM_INLINE; loom::noinline, loom::hot, and loom::cold separate placement requirements from profitability evidence Rename the attribute and macro. Use hot or cold only as profitability evidence; they do not replace explicit inline or noinline requirements.
C++ typed views loom::type::view<Shape, T, Role> used a loom::encoding::role third argument defaulting to layout loom::type::view<Shape, T, Encoding> uses a ranked loom::type::encoding<Role, Rank> type; omitting it denotes an unencoded view and does not infer a dense layout Replace explicit role arguments with the corresponding ranked encoding type. Add a layout encoding type to two-argument aliases that must match constructors carrying a layout; leave the argument omitted only for genuinely unencoded views.
Authored command IR and generated C API command.program.def, command.program.decl, and command.program.launch command.def, command.decl, and command.launch; generated C symbols use the same flattened vocabulary Rename source operations and generated C API calls. Serialized operation ordinals are unchanged.
Authored vector literal broadcasts vector.constant accepted an integer or floating-point splat attribute vector.splati carries integer, predicate, index, and offset literals; vector.splatf carries floating-point literals; operand-driven vector.splat is unchanged Replace each vector.constant with vector.splati or vector.splatf according to its result element domain. There is no compatibility spelling.
Text model cards pool_capacity combined logical addressability and model-local cache quota context_capacity limits one conversation; shared physical capacity supplies the elastic ceiling; optional memory_bytes tightens the pool-wide budget Remove pool_capacity. Set memory_bytes only when the deployment needs a budget below available shared capacity.
Attention K/V cache placement Retained key tiles and value tiles commonly occupy separate planes Pair the key and value fragments consumed together in one retained record, while preserving each operand's internal matrix-load order; the controlled fragmented-attention witness takes 10.3% less time with unchanged instruction and resource counts Attention authors should try the interleaved [K0|V0][K1|V1]... layout. Update appenders and every scalar/matrix consumer together, keep all native variants within an instance on one geometry, and create a fresh model instance after changing layout. Complete 128K requests were neutral within observed variation.
Finite model source ABI prepare_batch(state, stage_tags, ranges, prepared_bytes) received an eager packed copy; run(stage, input, output) completed every selected request prepare_batch receives ptr<finite.batch>, run receives retained state and the batch, and collect_batch commits successful progress; requests can continue across cohorts and retain device ranges and partial results Accept the borrowed batch in prepare_batch; call finite.batch_pack only when a packed copy is needed. Add state and batch arguments to run, add collect_batch, and finish or publish each member's next offer from collection.
Native host CPU eligibility loom_runtime_host_create(worker_count, compiler_options, allocator, out_host) let Task and asynchronous I/O workers independently derive placement from the thread that happened to create them loom_runtime_host_create accepts an optional iree_thread_affinity_set_t after worker_count, and the selected CPU budget is preserved across Task topology and later I/O worker creation; NULL captures the constructing thread's effective Linux/Android affinity once Insert NULL after worker_count to preserve launcher-imposed affinity, or pass an explicit nonempty processor set. Update direct callers of the lower thread-affinity, Task-topology, proactor, and runner APIs to carry the same eligibility set. Darwin and WebAssembly reject explicit hard CPU sets because neither platform exposes that placement contract.
Text and sequence epoch publication publish_epoch began with the completed plan and feedback arguments publish_epoch receives the retained control-state buffer first, allowing successful epochs to retain source-owned observations Prepend control_state: buffer to each text and sequence publication export and update its service wrapper. Existing trailing arguments keep their meanings.
Image model output Models returned planar F32 CHW RGB in [-1, 1]; the service quantized to RGB8 Models return row-major packed RGBA8, and the service preserves all four channels through readback and PNG encoding Fuse final quantization into the model, write four R/G/B/A bytes per pixel, and size output storage accordingly. Supply alpha explicitly.
Telemetry event consumers History event weight_activity parameter_activity, with the same fields and accepted-operation meaning Update event-name matching.
AMD GPU runtime and build selection amdgpu selected the ROCr-backed HAL; builds used IREE_HAL_DRIVER_AMDGPU and IREE_HAL_AMDGPU_TARGETS and coupled the driver to ROCm runtime headers amdgpu, rdna, and cdna select one libamdf-backed AMD GPU HAL; the legacy driver, legacy-amdgpu selector, and old build options are removed; IREE_ROCM_PATH independently selects optional compilation and interoperability tooling Keep ordinary --device=amdgpu commands. Replace the build options with IREE_HAL_DRIVER_AMD_GPU and IREE_HAL_AMD_GPU_TARGETS, and qualify the native HAL for deployments that depended on ROCr-specific behavior. There is no legacy runtime fallback.
Scalar HAL and HRX virtual-memory APIs iree_hal_allocator_virtual_memory_*, iree_hal_allocator_physical_memory_*, and their deprecated hrx_allocator_* projections managed reservations, opaque physical handles, mappings, and permissions one operation at a time Virtual memory uses ordinary pool-backed buffers, hrx_virtual_buffer_create, and batched queue mappings; the scalar functions, physical-memory handle types, allocator vtable entries, protection types, and mapping ledgers are removed Construct accepted memory pools and a virtual buffer, allocate backing with mapping-source usage, and submit explicit mapping batches. Captures containing retired scalar replay operations report them as unsupported.
AMD GPU active completion policy Active user queues observed coherent completion for 100 µs before blocking Actual progress renews an active observation budget of up to one second; idle queues still sleep and kernel queues use native events CPU-constrained or battery-sensitive deployments should select --wait_mode=blocking or set completion_spin_ns explicitly.
LoomC command-product construction Each loomc_cmd_program_product_options_t supplied .link_index directly A reusable loomc_cmd_compiler_t retains the frozen index and bounded prepared-source plans; product options supply .compiler Create one command compiler from the frozen link index, pass it to related product builds, and release it after all synchronous builds return. Set retained_byte_limit or zero retention when the default 64 MiB policy is unsuitable.
LoomC source-Low inspection Exact-version clients could call loomc_pass_program_create_from_native_source_low_pipeline with a native pipeline callback The public target-pipeline selector exposes LOOMC_TARGET_PIPELINE_KIND_SOURCE_LOW and LOOMC_TARGET_PIPELINE_KIND_SOURCE_LOW_ARTIFACTS through loomc_pass_program_create_from_target_pipeline Select the required source-Low boundary in loomc_target_pipeline_options_t and create the pass program through the public target pipeline. Remove the native callback bridge.
Repository-only Python frontends In-tree loom.importers.mlir, loom.importers.tilelang, and their managed environments coupled foreign compiler dependencies to the HRX repository; none shipped in the wheel The base wheel ships the dependency-free loom.importers construction API; source-specific frontends and dependencies live with their integrations Externalize MLIR or TileLang parsing and versioning, then construct canonical Loom modules through loom.importers. There is no compatibility stub for the removed unshipped modules.

Notable pull requests

View the complete v0.1.0 to v0.2.0 comparison.


Install and artifacts

Loom SDK 0.2.0 contains the compiler, runtime, command-line tools, and Python APIs for Linux x86-64 and Windows x86-64.

Install the optimized profile from PyPI:

python -m pip install hrx-loom==0.2.0

For use outside Python, download the standalone archive for your platform:

  • loom-serve-0.2.0-manylinux_2_39_x86_64.zip
  • loom-serve-0.2.0-win_amd64.zip

Each archive contains one loom-serve executable with the SDK target providers built in:

unzip loom-serve-0.2.0-manylinux_2_39_x86_64.zip
./loom-serve --version

For native AMD GPU/NPU discovery and activation checks without installing the SDK, use the standalone amdf-info archive:

  • amdf-info-0.2.0-manylinux_2_39_x86_64.zip
  • amdf-info-0.2.0-win_amd64.zip

Each contains one executable with libamdf embedded and requires neither ROCr nor XRT. amdf-info --format=json reports the native inventory; amdf-info --device=rdna --activate checks device acquisition and release.

Qualification

The attached wheels are the exact Release, Tracy, Debug, and ASAN profile artifacts qualified on Linux x86-64 and Windows x86-64. SHA256SUMS records the qualified wheels and standalone archives.