Minor Release 7.2.0

Pull Requests Since 7.1.0

Unlabeled

1. #265 - feat(ci): FPGA-less CI — emulator + GPU stub + regression suite + DKMS across 5 distros
2. #264 - Add GpuAsync user-space lifecycle API and refactor rdmaTest
3. #244 - Avoid allocations during runtime
4. #267 - Fix bring-up / teardown bugs in data_dev, aximemorymap, axistreamdma
5. #266 - include/GpuAsyncUser.h: Add offset methods for latency registers

Pull Request Details

Avoid allocations during runtime

Author:	Jeremy L. lorelli@slac.stanford.edu
Date:	Wed Apr 29 12:29:43 2026 -0700
Pull:	#244 (206 additions, 111 deletions, 3 files changed)
Branch:	slaclab/pr-no-allocs

Notes:

Description

Pre-allocates scratch buffers so we don't have to keep calling kmalloc during dmaReadIndex and such.

Introduces the following limitations:

• Max number of indices that can be returned at once is cfgRxCount + cfgTxCount
• Max number of buffers that can be read or written at once is cfgRxCount + cfgTxCount

If those conditions are violated, the ioctl returns with error.

I am not sure if the DAQ is trying to read/write/return more than its allotted amount.

---

Add GpuAsync user-space lifecycle API and refactor rdmaTest

Author:	Larry Ruckman ruckman@slac.stanford.edu
Date:	Wed Apr 29 15:59:53 2026 -0700
Pull:	#264 (996 additions, 317 deletions, 12 files changed)
Branch:	slaclab/gpu-api-dev

Notes:

Summary

• Add GpuAsyncContext + gpuAsyncInit()/gpuAsyncCleanup() to GpuAsyncUser.h, encapsulating the full GPU/FPGA bring-up and strict reverse-order tear-down for GPUDirect RDMA applications
• Refactor rdmaTest.cu to use the new API (replaces ~185 lines of inline boilerplate in main() and threads GpuAsyncContext* through runSimpleLoop)
• Harden ioctl wrappers in GpuAsync.h: gpuGetGpuAsyncVersion and gpuGetMaxBuffers now return ssize_t so -1/errno failures are representable; gpuIsGpuAsyncSupported correctly maps ioctl failure to false; doc comments reflect -1/errno (not -ENOTSUPP)
• CUDA 13.2 compatibility via #if CUDA_VERSION >= 13000 guard for the cuCtxCreate 3->4 arg change; make GpuAsyncUser.h self-contained
• Add Doxygen comments across the public API and a Sphinx reference page (docs/reference/gpu-async-api.rst) covering both GpuAsync.h and GpuAsyncUser.h, rendered via Breathe

Lifecycle correctness

• gpuAsyncInit: goto-based unwind where each label unwinds exactly the steps that succeeded; seeds ctx->fd = -1 before any early-return path; disables FPGA DMA engines early; flushes the GPU stream before enabling engines; checks cuStreamWriteValue32 return; logs errno / CUDA error names on syscall and CUDA failures; caches gpuGetMaxBuffers / gpuGetGpuAsyncVersion; validates bufCnt > 0 and bufSize > 0 before alignment/bound checks; NULL-guards every cudaFree in unwind loops and unwinds partial NVIDIA memory registration on rx failure
• gpuAsyncCleanup: strict reverse-init order; NULL-guard every handle; fd = -1 after teardown to prevent close(stdin) on double cleanup; clears ctx->devRegs and stale handles
• Init-failure exit code normalized to 1

rdmaTest fixes

• Validate hdr.size before loopback rx->tx copy and against allocated buffer on file dump
• Enqueue loopback device-to-device copy on ctx->cuStream (not the default stream)
• Bound tmpbuf / -x file-dump allocation to actual payload size using size_t to match the -x path; clamp dump copy to per-buffer cudaMalloc size; guard negative s_dumpBytes before vector sizing
• Use cached ctx->dmaHeaderSize in the loopback hot loop copy
• Cast bufSize to unsigned for %x; dedupe/reorder includes

Test plan

• make -C data_dev/app cuda compiles cleanly under CUDA 13.2 (NVCC) and CUDA 12.x
• ./rdmaTest -d /dev/datadev_0 -b 16 -s 0x100000 -c 100 -- receives events, exits cleanly
• Loopback ./rdmaTest ... -l -- round-trip with no data corruption
• ./rdmaTest ... -x <file> -- file dump bounded correctly

---

feat(ci): FPGA-less CI — emulator + GPU stub + regression suite + DKMS across 5 distros

Author:	Larry Ruckman ruckman@slac.stanford.edu
Date:	Wed Apr 29 13:30:45 2026 -0700
Pull:	#265 (18648 additions, 334 deletions, 112 files changed)
Branch:	slaclab/emu-fpga-ci-testing

Notes:

Summary

End-to-end FPGA-less CI for aes-stream-drivers. A virtual PCI device emulates the AXIS Gen2 DMA engine in 128-bit descriptor mode, an nvidia_p2p_* stub services the GPU Async V4 stack, and a regression suite drives the real datadev driver through DMA / GPU / ioctl / mmap / interrupt paths across a five-distro build+load+test matrix on every push — no FPGA hardware or GPUs required. A KVM-based local CI harness reproduces the cloud matrix bit-for-bit.

What's in the branch

Emulator kernel module — emulator/driver/

• Virtual PCI host bridge (pci_alloc_host_bridge) with full BAR0 register emulation: enableVer, addrWidth, channelCount, maxSize, online, DmaDataBytes.
• enableVer R/O field enforcement (emu_enforce_enablever_ro()) reasserts version, Desc128En, and enableCnt on every poll cycle so the driver's AxisG2_Clear full-word zero on rmmod cannot silently drop the next load into 64-bit mode.
• online 1→0 edge fully resets DMA-engine state (ring base regs, WriteFree shadows) — moved out of the production driver and into the emulator where the reset belongs.
• 128-bit descriptor DMA loopback engine preserves AXIS sideband (fuser/luser/cont/dest); seed-phase buffer discovery; IRQ coalescing (one IRQ per poll cycle) prevents WriteFree shadow drain.
• irq_work-driven virtual IRQ exercises the real ISR → workqueue → AxisG2_Process path.
• iomem_resource claim so the driver's request_mem_region succeeds against alloc_pages-backed BAR0.
• In-kernel PRBS LFSR + verifier; gpu_engine RX/TX FSMs and poll kthread.
• emu_gpu_poll kthread promoted to SCHED_FIFO(1) via sched_set_fifo_low(); emu_gpu_poll_interval_us defaults to 1000 µs (dev) and is overridden to 100 µs in CI so userspace doorbell spinners on a 2-vCPU runner cannot starve the wakeup.
• Lazy symbol_get + KBUILD_EXTRA_SYMBOLS for clean gpu_stub drain at unload; stray-DMA guard + RX-warn rate limit on rmmod.
• build_version sysfs attr + .build_version sideband file frozen at Makefile build time.

GPU stub module — emulator/gpu_stub/

• nvidia_p2p_stub.ko exports nvidia_p2p_* symbols (V4 only — V1/V2/V3 intentionally skipped).
• emu_gpu_addr_table.{c,h} — hashtable + miscdevice fops + EXPORT_SYMBOL_GPL; refcount-safe find_by_idx; lock-disciplined stub_try_drain (hash_empty under addr_ht_lock, synchronize_rcu outside lock).
• VMA-reuse branch in nvidia_p2p_get_pages (emu_gpu_addr_is_stub_vma).
• Same build_version sysfs surface as the emulator driver.
• Kernel shims for pre-6.8 (MAX_PAGE_ORDER) and pre-6.3 (vm_flags_set); RHEL 9.4+ exempt.

User-space test programs — data_dev/app/src/

• dmaIoctlTest — exercises 28 ioctls in 27 numbered checks (24 DMA + 2 AXIS + 1 AxiVersion + 1 GPU readiness; check 5 covers Get_Index+Ret_Index round-trip).
• dmaFileOpsTest — open / close / select / mmap / read.
• dmaErrorTest — buffer exhaustion, oversized write, invalid index.
• dmaGpuIoctlTest — all 6 GPU async ioctls.
• dmaGpuToggleTest — disable/enable pairs with countReset quiesce + per-frame PRBS preamble + maxBuffers + toggle_resume verification, retried on transient flakes.
• rdmaTestEmu — GPUDirect-style loopback with --sweep {64, 4096, 65536, 1M}, VA alignment, PRBS check, atexit cleanup, signed-int CLI validation, retry.
• dmaSmallFrameTest — 1-4 byte payload loopback with select() drain (closes the count==0 vacuous-PASS hazard).

Test harness — tests/, scripts/ci/

• Phase-1 load + basic DMA: test_proc, test_params, test_data_integrity, test_dma_rate, test_multichannel, test_reload_segfault.
• Phase-3 gap coverage: test_idx_loopback, test_frame_sizes, test_backpressure, test_irq_modes, test_tuser_sweep, test_concurrent_open, test_small_frames.
• Test-5 reload cycle gates cfgMode=2, cfgBgThold, and rmmod-under-load.
• GPU phase runs only the GPU-unique tests (test_gpu_ioctls, test_gpu_proc, test_gpu_dma_loopback) plus the nvidia_p2p_stub drain_cb load/unload-order check; CPU coverage is provided once by test-cpu.sh and not re-run.
• Modular CI scripts (scripts/ci/build-{cpu,gpu}.sh, load-modules-{cpu,gpu}.sh, test-{cpu,gpu}.sh, test-dkms-{cpu,gpu}{,-smoke}.sh, unload-modules-{cpu,gpu}.sh, check-dmesg.sh, install-deps.sh).
• check-dmesg.sh matches canonical kernel tokens, scopes to the subtest window, verifies the kmsg baseline marker landed, and rejects vacuous PASS under dmesg_restrict=1.
• timeout-wrapped insmod across test-gpu, reload_segfault, and the deterministic test binaries to prevent CI hang on rmmod-under-load oops.
• Hardened against set -e pitfalls (((FAILED++)), grep -q && echo || true, pre-increment), retry around known-flaky cells (dmaLoopTest, irq_modes subtests, sweep/soak), printf '%s\n' instead of echo for arbitrary content (DELTA / DMESG_DELTA / STATUS_RESP / KVER).

CI pipeline — .github/workflows/ci_pipeline.yml

• Single workflow replacing aes_ci.yml. Five distros run the full build + load + test + DKMS sequence in both CPU and GPU phases: ubuntu:22.04, ubuntu:24.04, rockylinux:9, debian:experimental, fedora:rawhide.
• Host kernel headers bind-mounted into containers; load/test gated at runtime on CI_HOST_MATCH=1, otherwise falls back to dkms ldtarball --no-prepare-kernel smoke.
• Kbuild compatibility layer: when host scripts/ binaries (modpost, gendwarfksyms, objtool) are glibc-incompatible, recompile from source with ldd/find -L detection and selective stubbing; scripts/include on the recompile path for older distros.
• ensure_kernel_gcc GCC-toolset fallback with flag validation, awk-based version extraction (no PCRE), and gcc-N symlink when gcc-toolset is unavailable; rejects gcc-N binaries whose reported major doesn't match.
• Phase-1 doc-build smoke test (Sphinx + Doxygen + HTML validation).
• gpu_async.c merge-base zero-diff gate — production driver source must not drift under the emulator.
• Step summaries, error annotations, diagnostic-artifact upload on failure.

Local CI — scripts/ci-local/, run_local_ci.sh, scripts/run_ci_parity.sh

• KVM runner with virtiofs shared FS, cloud-init VMs matching the cloud kernel.
• Per-cell runner + multi-distro orchestrator with parallel KVM mode (5+5 concurrent cells), diagnostic capture, hang repro.
• run_matrix.sh parallel-dispatch fix: explicit VM_SLOT=$((VM_SLOT + 1)) (the ((VM_SLOT++)) post-increment returned 0 on the first iteration and tripped set -e, silently dropping all remaining load-test cells).
• QEMU/TCG fallback for environments without KVM; QEMU/TCG path loads nvidia_p2p_stub before datadev_emulator to match the cloud sequence.
• AI_LOCAL_CI_TESTING.md and LOCAL_CI_TESTING.md runbooks.

Documentation

• docs/explanation/ci-pipeline.rst — pipeline architecture, distro matrix, emulator GPU poll thread (SCHED_FIFO + tick cadence), and enableVer R/O field enforcement.
• docs/reference/test-applications.rst — user-space test program reference with 28-ioctl breakdown.
• docs/how-to/dkms-install.rst — expanded multi-distro install (Ubuntu/Debian, Rocky/RHEL 9, CentOS 7, RHEL with subscription).
• include/AxiVersion.h and include/GpuAsync.h kernel-doc converted to Doxygen @param / inline /**<; gpuIsGpuAsyncSupported returns bool correctly via > 0 (signed-bool false-positive on -1 fixed).
• README adds host/guest kernel-match note.

Key design decisions

• 128-bit descriptor mode only — 64-bit mode is deprecated; the driver only creates a workqueue and processes completions when Desc128En=1.
• GPU Async V4 only — V1/V2/V3 register layouts skipped.
• addrWidth=1 (2 ring entries) — serializes TX descriptors to prevent shadow-register overwrite race.
• One IRQ per poll cycle — coalesces forceInt and TX completion to keep WriteFree shadows from draining the free pool between polls.
• Seed phase for buffer discovery — zero-size RX completions probe valid buffer indices when init-time WriteFree capture is lossy.
• BUFF_STREAM (cfgMode=2) in CI — avoids CMA allocation failures on runners.
• enableVer R/O fields enforced every poll cycle — mirrors the VHDL so AxisG2_Clear rmmod-time zero cannot silently land the next load in 64-bit mode.
• SCHED_FIFO(1) for emu_gpu_poll — RT priority keeps usleep_range wakeups punctual under CFS contention from userspace doorbell spinners; CI tightens cadence to 100 µs.
• GPU phase runs only GPU-unique tests — CPU coverage runs once.
• Kbuild recompile-from-source — keeps older-glibc distros in the matrix without host-kernel lock-in.

Test plan

• Phase 2 (CPU) green on all five distros: ubuntu:22.04, ubuntu:24.04, rockylinux:9, debian:experimental, fedora:rawhide.
• Phase 3 (GPU) green on all five distros.
• DMA loopback PRBS integrity holds across the 30s-per-cell sustained run.
• All Phase-1 + Phase-3 tests pass (proc, params, data_integrity, dma_rate, multichannel, reload_segfault, idx_loopback, frame_sizes, backpressure, irq_modes, tuser_sweep, concurrent_open, small_frames).
• GPU-unique tests pass: test_gpu_ioctls, test_gpu_proc, test_gpu_dma_loopback.
• dmaGpuToggleTest passes across state transitions with frame-level PRBS verification.
• rdmaTestEmu --sweep passes across {64, 4096, 65536, 1M}.
• Test-5 reload cycles (cfgMode=2, cfgBgThold, rmmod-under-load) complete without timeout.
• Driver reload after rmmod keeps Desc128En=1 (no silent 64-bit fallback).
• gpu_async.c merge-base zero-diff gate stays green.
• DKMS smoke passes on non-CI_HOST_MATCH cells; full DKMS install/remove passes on matching cells (CPU + GPU).
• Phase-1 doc build smoke passes.
• aes_ci.yml removed; ci_pipeline.yml preserves required-check job keys.

---

include/GpuAsyncUser.h: Add offset methods for latency registers

Author:	Jeremy L. lorelli@slac.stanford.edu
Date:	Mon Apr 27 21:18:22 2026 -0700
Pull:	#266 (43 additions, 3 deletions, 1 files changed)
Branch:	slaclab/pr-gpu-async-user-update

Notes:

Description

Adds a version override parameter to the constructor for cases where you want to override the GpuAsyncCore version

Adds methods for getting the offsets for the latency registers, for use with CUDA kernels.

---

Fix bring-up / teardown bugs in data_dev, aximemorymap, axistreamdma

Author:	Larry Ruckman ruckman@slac.stanford.edu
Date:	Wed Apr 29 13:57:50 2026 -0700
Pull:	#267 (77 additions, 26 deletions, 6 files changed)
Branch:	slaclab/datadev-init-cleanup

Notes:

Summary

Reviewed and fixed bring-up (probe / _Init) and teardown (remove / _Exit) sequences for three kernel drivers and the shared dma_common helpers they call:

• data_dev/driver/src/data_dev_top.c (PCIe datadev driver — symlink to common/driver/data_dev_top.c)
• Yocto/recipes-kernel/aximemorymap/files/aximemorymap.c
• Yocto/recipes-kernel/axistreamdma/files/axistreamdma.c

While in there, also fixed a pre-existing version-guard bug that was preventing builds against kernel 6.2 / 6.3.

Bugs fixed

Resource leaks on probe failure paths

• data_dev_top.c: DataDev_Probe failures after Dma_MapReg succeeded (IRQ-missing check, dma_set_mask / dma_set_coherent_mask failures) only called pci_disable_device and leaked the iomap + mem_region claim. Added err_unmap label that calls Dma_UnmapReg before disabling the PCI device.
• axistreamdma.c: Rce_Probe failures after irq_of_parse_and_map and Dma_MapReg used bare return -1 and leaked the virtual IRQ mapping plus the ioremap + mem_region claim. Added err_post_irq_map / err_post_mapreg / err_dma_init labels that unwind in reverse order, including irq_dispose_mapping and a memset to clear the gDmaDevices slot.

Teardown bugs

• aximemorymap.c: Map_Exit was missing cdev_del(&dev.charDev), leaving the character device live across module unload. Teardown order also did not match the reverse of Map_Init (class_destroy ran after unregister_chrdev_region). Reordered to: cdev_del -> device_destroy -> class_destroy -> unregister_chrdev_region -> free maps.
• axistreamdma.c: Rce_Remove never called irq_dispose_mapping on the virtual IRQ created by irq_of_parse_and_map — every probe -> remove cycle leaked an IRQ mapping. (Dma_Clean is intentionally untouched so the PCI path through data_dev_top, which uses pdev->irq directly, is unaffected.)

API-correctness bugs in helper code

• dma_common.c / aximemorymap.c: class_create() and device_create() return ERR_PTR on failure, not NULL. The existing == NULL checks let real failures through to a kernel oops on gCl->devnode = .... Switched to IS_ERR_OR_NULL(). On class_create failure, also reset gCl = NULL so the shared (gDmaDevCount == 0 && gCl != NULL) cleanup logic stays safe.
• dma_common.c / aximemorymap.c: cdev_add() returns -errno on failure (-EBUSY, -ENOMEM, …), but the checks were == -1 and only caught -EPERM. Switched to < 0.
• dma_common.c: Dma_UnmapReg was non-idempotent (no NULL check, no clear of dev->base) and Dma_MapReg left dev->base pointing at unmapped memory after a request_mem_region failure. Made Dma_UnmapReg idempotent and Dma_MapReg clear dev->base on error so multi-path cleanup is safe.

Kernel version guards

• axistreamdma.c / axistreamdma.h: the platform-driver .remove callback returns int prior to kernel 6.11 and void from 6.11 onward. The current void Rce_Remove(...) only matched 6.11+, so the driver did not compile on the supported kernel range (3.10 - 6.10). Version-guarded both the prototype and the definition.
• dma_common.c / dma_common.h / aximemorymap.c: the class.devnode() callback became const struct device * in upstream commit ff62b8e6588f ("driver core: make struct class.devnode() take a const *") in v6.2-rc1, not v6.4. The version guard for Dma_DevNode / Map_DevNode was conflated with the separate class_create() THIS_MODULE-removal change in v6.4 (commit 1aaba11da9aa), causing a function-pointer type mismatch when building against kernels 6.2 and 6.3. Lowered only the devnode guard to KERNEL_VERSION(6, 2, 0) (the class_create() guard at 6.4 stays — that change really is in 6.4). Kept the RHEL_RELEASE_VERSION(9, 4) fallback for both, matching how RHEL 9.4 backported the upstream changes.

Commits

Five atomic commits, smallest blast radius first:

1. dma_common: harden init/cleanup error paths and helper APIs
2. data_dev_top: unmap registers on probe failures after Dma_MapReg
3. aximemorymap: fix Map_Exit teardown order and init error checks
4. axistreamdma: clean up probe/remove resource paths and guard remove signature
5. dma_common, aximemorymap: lower class.devnode const guard from 6.4 to 6.2

Test plan

• cpplint (matching CI invocation) clean on all modified files
• CI whitespace / tab checks clean
• Local make driver against all installed kernels: 5.15.0-91, 5.15.0-139, 5.19.0-50, 6.2.0-39, 6.5.0-45, 6.8.0-59, 6.8.0-107, 6.8.0-110 — all build clean. (Previously 6.2.0-39 failed with -Werror=incompatible-pointer-types.)
• Live smoke test on a real /dev/datadev_0: two rmmod datadev + insmod datadev.ko cycles on running 6.8.0-110 kernel — module reloaded cleanly each time, /dev/datadev_0 re-created, IRQ 162 reacquired, no warnings / oops / leak messages in dmesg, /proc/iomem shows a single f3000000-f3ffffff : datadev_0 claim with no duplication.
• CI build matrix (Ubuntu 22.04 / 24.04 / Rocky 9 / Debian experimental / CentOS 7) — to verify on PR
• Yocto / platform-driver path — needs target hardware to validate axistreamdma rmmod/insmod cycle and confirm IRQ-mapping leak fix

Notes for review

• All fixes preserve existing behavior on the success path; only error / teardown paths change.
• No public API change. The only internal-helper signature change is Dma_UnmapReg becoming idempotent (no callers needed updating).
• Dma_Clean is left unchanged — the PCI path through data_dev_top uses pdev->irq directly and never calls irq_of_parse_and_map, so disposing the mapping there would be wrong. Only axistreamdma's Rce_Remove does it explicitly.
• Sister drivers rce_stream/driver/src/rce_top.c and rce_hp_buffers/driver/src/rce_top.c likely share the same IRQ-mapping leak and inverse .remove signature problem (int -> breaks on 6.11+). Out of scope for this PR.

---