You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The ibverbs raw-RX benchmark on main fails during daqiri_init() while creating the DevX receive queue. The same raw-RX configuration initializes successfully with an image built before #281.
The failure is an initialization regression, not a throughput result:
#281 made the RQ request MLX5_RQC_TIMESTAMP_FORMAT_REAL_TIME unconditionally. The affected device does not advertise MLX5DV_CONTEXT_FLAGS_REAL_TIME_TS, and its firmware rejects that unsupported RQ timestamp format.
Individually restoring the other RX-side changes did not resolve the failure:
Restoring the previous striding-RQ geometry still fails with syndrome 0x43d98f.
Restoring the previous RX WQ UMEM access flags still fails with syndrome 0x43d98f.
Restoring both still fails.
Omitting only the unsupported real-time timestamp request allows the identical configuration to initialize.
On the affected ARM64 system, run daqiri_bench_raw_gpudirect in RX-only mode with a valid single-port raw ibverbs configuration using a GPU memory region and one striding RX queue.
Observe that daqiri_init() fails at DevX CREATE_RQ before traffic or throughput measurement begins.
Run the same configuration with a pre-ibverbs improvements #281 image and observe successful initialization.
No machine-local interface names, PCI addresses, MAC addresses, network addresses, or hostnames are required to reproduce the issue; substitute locally valid values in the example raw-RX configuration.
Expected behavior
Raw ibverbs RX initialization should create the DevX RQ successfully on devices that do not support real-time CQ timestamps. DAQIRI should request the real-time format only when mlx5dv_query_device() advertises MLX5DV_CONTEXT_FLAGS_REAL_TIME_TS; otherwise it should retain the device-clock CQ format and convert timestamps through the existing clock-conversion path.
The valid #281 changes should remain intact: RX WQ UMEM registration with no remote-access permissions, the reduced 64-stride region geometry, and the adapter-relative log_wq_pg_sz encoding.
Environment overview (please complete the following information)
Environment location: bare-metal host; DAQIRI runs in the privileged project container
Method of DAQIRI install: project container built from source
The mlx5 WQ page-size field is encoded relative to the adapter's fixed 4-KiB page. It is distinct from the page size used to align/register the UMEM and was not the cause in the individual-revert matrix. Increasing the buffer count to restore the previous effective queue depth also does not fix initialization.
A narrow fix will capability-gate the RQ timestamp format and select the matching timestamp decode path per queue. Hardware reproduction is required because this failure occurs in firmware during DevX object creation.
Describe the bug
The ibverbs raw-RX benchmark on
mainfails duringdaqiri_init()while creating the DevX receive queue. The same raw-RX configuration initializes successfully with an image built before #281.The failure is an initialization regression, not a throughput result:
#281 made the RQ request
MLX5_RQC_TIMESTAMP_FORMAT_REAL_TIMEunconditionally. The affected device does not advertiseMLX5DV_CONTEXT_FLAGS_REAL_TIME_TS, and its firmware rejects that unsupported RQ timestamp format.Individually restoring the other RX-side changes did not resolve the failure:
0x43d98f.0x43d98f.Steps/Code to reproduce bug
Build current
mainin the project container with:BASE_TARGET=dpdk DAQIRI_ENGINE="dpdk ibverbs" scripts/build-container.shOn the affected ARM64 system, run
daqiri_bench_raw_gpudirectin RX-only mode with a valid single-port raw ibverbs configuration using a GPU memory region and one striding RX queue.Observe that
daqiri_init()fails at DevXCREATE_RQbefore traffic or throughput measurement begins.Run the same configuration with a pre-ibverbs improvements #281 image and observe successful initialization.
No machine-local interface names, PCI addresses, MAC addresses, network addresses, or hostnames are required to reproduce the issue; substitute locally valid values in the example raw-RX configuration.
Expected behavior
Raw ibverbs RX initialization should create the DevX RQ successfully on devices that do not support real-time CQ timestamps. DAQIRI should request the real-time format only when
mlx5dv_query_device()advertisesMLX5DV_CONTEXT_FLAGS_REAL_TIME_TS; otherwise it should retain the device-clock CQ format and convert timestamps through the existing clock-conversion path.The valid #281 changes should remain intact: RX WQ UMEM registration with no remote-access permissions, the reduced 64-stride region geometry, and the adapter-relative
log_wq_pg_szencoding.Environment overview (please complete the following information)
BASE_TARGET=dpdk DAQIRI_ENGINE="dpdk ibverbs" scripts/build-container.shEnvironment details
6.17.0-1014-nvidia15b3:1021)28.45.40282407mlnx52-1.2407061maincontainingdbf0c8b(regression); pre-ibverbs improvements #281 image succeedsAdditional context
The mlx5 WQ page-size field is encoded relative to the adapter's fixed 4-KiB page. It is distinct from the page size used to align/register the UMEM and was not the cause in the individual-revert matrix. Increasing the buffer count to restore the previous effective queue depth also does not fix initialization.
A narrow fix will capability-gate the RQ timestamp format and select the matching timestamp decode path per queue. Hardware reproduction is required because this failure occurs in firmware during DevX object creation.