v1.3.2

1.3.2

Summary

NIXL 1.3.2 is a targeted patch release that fixes a cross-node EFA transfer hang introduced in 1.3.1. When the progress thread is enabled, NIXL posts each WRITE descriptor under a separate endpoint lock while using FI_MORE to keep a 16-descriptor batch open; on efa/efa-direct providers, CQ progress can acquire that same endpoint lock before the closing post, leaving the batch permanently open and stalling the transfer. An initial fix disabling FI_MORE on EFA (#1924) was replaced by a more targeted per-rail flush strategy (#1989) that preserves batching throughput on all providers.

The fix introduces per-rail flush tracking: a precompute pass identifies the last descriptor on each rail before posting begins, and FI_MORE is cleared only at that descriptor (or when the per-rail batch reaches NIXL_LIBFABRIC_FI_MORE_BATCH_SIZE). This guarantees every rail's batch closes without a fixed-width group boundary assumption and without disabling batching globally.

Bugfixes

• [Libfabric] Fix cross-node EFA transfer hang with progress thread: On efa and efa-direct providers, CQ progress can acquire the endpoint lock between a batch's first and closing FI_MORE post, preventing the batch from ever being flushed. NIXL 1.3.1 users with the progress thread enabled would see decode workers stuck in KVPoll.WaitingForInput returning empty responses. An intermediate fix that disabled FI_MORE entirely on EFA (#1924) was reverted (#1984) in favour of the per-rail flush in #1989, which closes each rail's batch correctly without giving up batching throughput on non-EFA providers. (#1924, #1984, #1989)

Known Issues

Full Changelog: 1.3.1...75ead3d7