#285 - Add direct polling latency benchmark and BlueFlame posting

AGENTS.md

@@ -37,6 +37,7 @@ There is no unit test suite. Verification is done via the benchmark executables
 | Executable | Source | Typical config |
 |---|---|---|
 | `daqiri_bench_raw_gpudirect` | `raw_gpudirect_bench.cpp` | `daqiri_bench_raw_tx_rx.yaml`, `daqiri_bench_raw_tx_rx_4q.yaml`, `daqiri_bench_raw_tx_rx_spark.yaml`, `daqiri_bench_raw_{tx,rx}_spark_xhost.yaml`, `daqiri_bench_raw_sw_loopback.yaml`, `daqiri_bench_raw_hw_loopback_ibverbs.yaml`, `daqiri_bench_raw_rx_multi_q.yaml`, `daqiri_bench_raw_tx_rx_vxlan.yaml`, `daqiri_bench_raw_tx_rx_vlan.yaml`, `daqiri_bench_raw_tx_rx_gre.yaml`, `daqiri_bench_raw_tx_rx_nvgre.yaml`, `daqiri_bench_raw_tx_rx_spark_mq.yaml` (mq base; `run_spark_mq_bench.sh` derives the 4 cells via `scripts/gen_spark_mq_config.py`), `daqiri_bench_raw_tx_rx_pacing.yaml` (per-queue `pacing_mbps`; DPDK engine only) |
+| `daqiri_bench_raw_latency` | `raw_latency_bench.cpp` | `daqiri_bench_raw_latency_ibverbs.yaml` — caller-driven direct TX/RX, RX hardware timestamps, 64–8192-byte power-of-two latency sweep |
 | `daqiri_example_dynamic_rx_flow` | `dynamic_rx_flow_example.cpp` | `daqiri_example_dynamic_rx_flow.yaml` — `flow_isolation: true` startup followed by runtime scalar queue steering, multi-queue RSS, and raw-engine decap/pop flow add/delete |
 | `daqiri_bench_raw_hds` | `raw_hds_bench.cpp` | `daqiri_bench_raw_tx_rx_hds.yaml` |
 | `daqiri_bench_raw_reorder_seq` | `raw_reorder_seq_bench.cpp` | `daqiri_bench_raw_tx_rx_reorder_seq_1024*.yaml`, `daqiri_bench_raw_rx_reorder_seq_*.yaml` |

docs/benchmarks/raw_benchmarking.md

@@ -238,6 +238,63 @@ After having modified the configuration file, ensure you have connected an SFP c
 
 By default the application runs for 10 seconds and then exits. You can change the duration by passing `--seconds <N>` after the YAML path, or stop it gracefully at any time with `Ctrl-C`.
 
+### Direct-polling latency sweep
+
+`daqiri_bench_raw_latency` measures the caller-driven raw ibverbs path with one packet
+outstanding at a time. Its template, `daqiri_bench_raw_latency_ibverbs.yaml`, configures both TX
+and RX with `poll_mode: direct`, enables per-packet RX hardware timestamps, and uses host-pinned
+buffers by default. The benchmark also supports `huge` and `device` packet memory: packet setup and
+identity checking use `cudaMemcpyDefault`, with both copies deliberately outside the reported
+`send_tx_burst()` call to `get_rx_burst()` return interval. It
+sweeps 64, 128, 256, ..., 8192-byte L2 frames; sizes exclude the Ethernet FCS added by the NIC.
+
+Replace the TX/RX PCI BDFs, master/application cores, destination MAC, and RX port's
+`ptp_device` in the template. The destination MAC must be the receiving port's MAC. Connect the
+physical loopback path, then run:
+
+```bash
+sudo ./build/examples/daqiri_bench_raw_latency \
+  examples/daqiri_bench_raw_latency_ibverbs.yaml \
+  --samples 10000 --warmup 1000 --csv latency.csv \
+  --realtime-priority 90
+```
+
+For a single-port internal comparison, merge the TX and RX queues under one interface, set both
+`bench_latency` interface names to it, use that port's MAC and PTP device, and set
+`daqiri.cfg.loopback: "hw"`. This retains the same timing boundaries while replacing the physical
+cable/peer-port return with the mlx5 hardware self-loopback path.
+
+`--realtime-priority N` locks current and future mappings with `mlockall()` and verifies that the
+latency thread is running under `SCHED_FIFO` at priority `N`. It requires the corresponding
+realtime scheduling and memory-lock privileges; omit it to retain normal `SCHED_OTHER` behavior.
+
+For each packet, the measured path follows this timeline:
+
+```text
+t0  Application timestamps immediately before send_tx_burst()
+t1  send_tx_burst() returns
+t2  NIC records the packet's RX hardware timestamp
+t3  get_rx_burst() returns the matching packet to the application
+```
+
+The summary and optional per-sample CSV combine those points into these boundaries:
+
+| Column | Start | End | What it includes |
+|--------|-------|-----|------------------|
+| `tx_call_to_return_ns` | `t0` | `t1` | Direct WQE construction and SQ doorbell submission |
+| `tx_call_to_rx_hw_ns` | `t0` | `t2` | TX submission, NIC transmit, physical or hardware-loopback return, and NIC receive |
+| `rx_hw_to_app_ns` | `t2` | `t3` | CQ visibility, direct polling, and API return |
+| `tx_call_to_app_ns` | `t0` | `t3` | Complete measured application round trip |
+
+`tx_call_to_rx_hw_ns` is a loopback-ingress proxy, not an actual TX egress timestamp: DAQIRI does
+not currently expose a TX hardware timestamp, so this value also contains the cable and NIC RX
+latency. When `ptp_device` is set, the benchmark uses Linux's non-mutating extended
+PHC/system cross-timestamp ioctl before and after each size and interpolates the offset for every
+sample. It prints the maximum measured cross-clock uncertainty. Without `ptp_device`, the NIC PHC
+must be synchronized with `CLOCK_REALTIME`; negative or implausibly large values indicate a clock
+problem. Pin the application core to an isolated physical core, use the performance governor, and
+save the raw CSV when investigating tails.
+
 ### Single-port hardware loopback without a cable
 
 On mlx5 systems where the NIC remains available without a cable, DAQIRI can send packets back to

docs/tutorials/configuration-walkthrough.md

@@ -59,6 +59,11 @@ For a shorter selection guide, start with the [Benchmarking overview](../benchma
     - **TCP**: [`daqiri_bench_socket_tcp_tx_rx.yaml`](https://github.com/nvidia/daqiri/blob/main/examples/daqiri_bench_socket_tcp_tx_rx.yaml).
     - **TCP, DGX Spark netns wire loopback** (combined base): [`daqiri_bench_socket_tcp_tx_rx_spark_netns.yaml`](https://github.com/nvidia/daqiri/blob/main/examples/daqiri_bench_socket_tcp_tx_rx_spark_netns.yaml). Carries both roles. `examples/run_spark_bench.sh` (via `scripts/gen_spark_netns_config.py`) splits it per role and runs each in its own network namespace (`--mode server` / `--mode client`). See [Socket and RDMA Benchmarking](../benchmarks/socket_benchmarking.md#run-the-linux-socket-benchmark).
 
+??? question "I want to measure direct-polling packet latency"
+    Use [`daqiri_bench_raw_latency_ibverbs.yaml`](https://github.com/nvidia/daqiri/blob/main/examples/daqiri_bench_raw_latency_ibverbs.yaml) with `daqiri_bench_raw_latency`. It sweeps one outstanding packet at a time from 64 through 8192 L2 bytes, using powers of two, with both queues in caller-driven `poll_mode: direct`.
+
+    The benchmark reports direct `send_tx_burst()` call-to-return time, submit-to-hardware-RX time, hardware-RX-to-`get_rx_burst()` return time, and total submit-to-application time. It requires a physical loopback path, raw `ibverbs`, RX hardware timestamps, host-pinned buffers, and PTP synchronization between the NIC and `CLOCK_REALTIME`. See [Direct-polling latency sweep](../benchmarks/raw_benchmarking.md#direct-polling-latency-sweep) for the measurement boundaries and run command.
+
 ??? question "2. I have out-of-order UDP packets that need to be reordered on the GPU"
     DAQIRI's flagship pipeline: a CUDA kernel reads a sequence number from each packet's header and places packets at the correct offset in a GPU buffer, so a downstream consumer sees a fully ordered stream without a CPU touch. Configs run on `daqiri_bench_raw_reorder_seq` unless 2.4 applies. Sub-questions:
 

examples/CMakeLists.txt

@@ -50,6 +50,7 @@ set(DAQIRI_BENCH_CONFIGS
   daqiri_bench_raw_rx_multi_q.yaml
   daqiri_bench_raw_sw_loopback.yaml
   daqiri_bench_raw_hw_loopback_ibverbs.yaml
+  daqiri_bench_raw_latency_ibverbs.yaml
   daqiri_example_dynamic_rx_flow.yaml
   daqiri_example_gds_write_sw_loopback.yaml
   daqiri_example_gds_write_tx_rx.yaml
@@ -98,6 +99,12 @@ endfunction()
 
 add_daqiri_raw_bench(daqiri_bench_raw_hds raw_hds_bench.cpp)
 add_daqiri_raw_bench(daqiri_bench_raw_gpudirect raw_gpudirect_bench.cpp)
+add_executable(daqiri_bench_raw_latency raw_latency_bench.cpp)
+link_daqiri_bench(daqiri_bench_raw_latency)
+target_link_libraries(daqiri_bench_raw_latency PRIVATE CUDA::cudart)
+set_target_properties(daqiri_bench_raw_latency PROPERTIES
+  BUILD_RPATH "$ORIGIN/../src;$ORIGIN/../src/third_party/yaml-cpp"
+)
 if(DAQIRI_ENABLE_OTEL_METRICS)
   find_package(opentelemetry-cpp CONFIG QUIET COMPONENTS sdk exporters_prometheus)
   if(TARGET opentelemetry-cpp::prometheus_exporter)
@@ -170,6 +177,7 @@ endforeach()
 install(TARGETS
         daqiri_bench_raw_hds
         daqiri_bench_raw_gpudirect
+        daqiri_bench_raw_latency
         daqiri_bench_raw_reorder_seq
         daqiri_bench_raw_reorder_quantize
         daqiri_example_dynamic_rx_flow

examples/README.md

@@ -4,6 +4,7 @@ Standalone benchmark applications for testing performance of DAQIRI with various
 
 - `daqiri_bench_raw_gpudirect`: raw TX/RX with one device-memory packet segment
 - `daqiri_bench_raw_hds`: raw TX/RX with header-data split
+- `daqiri_bench_raw_latency`: direct-polling single-packet TX/RX latency sweep
 - `daqiri_bench_raw_reorder_seq`: raw RX sequence-number reorder benchmark
 - `daqiri_bench_raw_reorder_quantize`: raw RX sequence reorder with payload conversion
 - `daqiri_example_pcap_writer`: RX pcap writer with optional GPUDirect demo TX traffic
@@ -57,6 +58,7 @@ Run:
 ./build/examples/daqiri_bench_raw_gpudirect ./build/examples/daqiri_bench_raw_tx_rx.yaml --seconds 10
 ./build/examples/daqiri_bench_raw_gpudirect ./build/examples/daqiri_bench_raw_tx_rx_4q.yaml --seconds 10
 ./build/examples/daqiri_bench_raw_hds ./build/examples/daqiri_bench_raw_tx_rx_hds.yaml --seconds 10
+./build/examples/daqiri_bench_raw_latency ./build/examples/daqiri_bench_raw_latency_ibverbs.yaml --samples 10000 --warmup 1000 --csv latency.csv
 ./build/examples/daqiri_bench_raw_reorder_seq ./build/examples/daqiri_bench_raw_tx_rx_reorder_seq_1024.yaml --seconds 10
 ./build/examples/daqiri_bench_raw_reorder_quantize ./build/examples/daqiri_bench_raw_tx_rx_reorder_quantize_seq_batch.yaml --seconds 10
 ./build/examples/daqiri_example_pcap_writer ./build/examples/daqiri_example_pcap_writer_sw_loopback.yaml /tmp/daqiri-capture.pcap --tx
@@ -89,6 +91,7 @@ Included configs:
 | `daqiri_bench_raw_tx_rx_4q.yaml` | `daqiri_bench_raw_gpudirect` |
 | `daqiri_bench_raw_sw_loopback.yaml` | `daqiri_bench_raw_gpudirect` |
 | `daqiri_bench_raw_hw_loopback_ibverbs.yaml` | `daqiri_bench_raw_gpudirect` |
+| `daqiri_bench_raw_latency_ibverbs.yaml` | `daqiri_bench_raw_latency` |
 | `daqiri_example_dynamic_rx_flow.yaml` | `daqiri_example_dynamic_rx_flow` |
 | `daqiri_example_gds_write_sw_loopback.yaml` | `daqiri_example_gds_write` |
 | `daqiri_example_gds_write_tx_rx.yaml` | `daqiri_example_gds_write` |

examples/daqiri_bench_raw_latency_ibverbs.yaml

@@ -0,0 +1,80 @@
+# Single-packet latency sweep for caller-driven ibverbs direct TX/RX.
+#
+# Replace both PCI BDFs, the application CPU, and destination MAC. Connect the
+# TX port to the RX port and use the RX port's MAC as eth_dst_addr. For a
+# single-port optical loop, merge the TX/RX sections under one interface and
+# use that interface name for both bench_latency names. This config deliberately
+# does not enable DAQIRI's internal `loopback: "hw"` mode.
+#
+# The benchmark sweeps L2 frame sizes 64, 128, ..., 8192 bytes (FCS excluded).
+# CLOCK_REALTIME and the NIC PTP clock must be synchronized before running.
+%YAML 1.2
+---
+daqiri:
+  cfg:
+    version: 1
+    stream_type: "raw"
+    engine: "ibverbs"
+    master_core: <3>
+    debug: false
+    log_level: "info"
+
+    memory_regions:
+    - name: "Latency_TX"
+      kind: "host_pinned"
+      affinity: 0
+      num_bufs: 1024
+      buf_size: 8192
+    - name: "Latency_RX"
+      kind: "host_pinned"
+      affinity: 0
+      num_bufs: 4096
+      buf_size: 8192
+
+    interfaces:
+    - name: "latency_tx_port"
+      address: <0000:00:00.0>
+      tx:
+        queues:
+        - name: "latency_tx"
+          id: 0
+          poll_mode: "direct"
+          memory_regions:
+          - "Latency_TX"
+          offloads:
+          - "tx_eth_src"
+    - name: "latency_rx_port"
+      address: <0000:00:00.1>
+      rx:
+        hardware_timestamps: true
+        flow_isolation: true
+        queues:
+        - name: "latency_rx"
+          id: 0
+          poll_mode: "direct"
+          memory_regions:
+          - "Latency_RX"
+        flows:
+        - name: "latency_flow"
+          id: 0
+          action:
+            type: queue
+            id: 0
+          match:
+            udp_src: 4096
+            udp_dst: 4096
+
+bench_latency:
+  tx_interface_name: "latency_tx_port"
+  rx_interface_name: "latency_rx_port"
+  tx_queue_id: 0
+  rx_queue_id: 0
+  cpu_core: <5>
+  # PTP device associated with latency_rx_port (see `ethtool -T <netdev>`).
+  # Enables non-mutating PHC/CLOCK_REALTIME cross-timestamp calibration.
+  ptp_device: </dev/ptpN>
+  eth_dst_addr: <00:00:00:00:00:00>
+  ip_src_addr: "192.0.2.1"
+  ip_dst_addr: "192.0.2.2"
+  udp_src_port: 4096
+  udp_dst_port: 4096