perf(portscan): AF_XDP plan Phase 2 PR 1 — io_engine dispatch refactor + scanner fork

[skullcrushercmd]

Summary

Phase 2 PR 1 of 4 of the AF_XDP integration plan (#65). This PR ships:

1. A fork of the scanner C source at AnyVM-Tech/anyscan-engine-c so we
   can carry the AF_XDP integration patches without depending on third-party
   upstream cooperation. The fork is initialized from
   Lorikazzzz/VulnScanner-zmap-alternative- per plan docs(plans): AF_XDP integration plan for higher pps (Phase 1) #65 §9.1.
2. A vtable-based io_engine dispatch refactor in the scanner's
   src/engine.c, replacing the hardcoded
   pthread_create(..., sender_thread, ...) with a small indirection that
   resolves --io-engine={af_packet,pfring_zc,af_xdp} to the right
   per-thread init + tx/rx thread bodies.
3. A --io-engine CLI flag in src/conf.c (default af_packet,
   backwards-compatible).
4. A side-effect fix for the PF_RING ZC dispatch wart noted by anygpt-34
   in plan §2.3: with this PR, when the scanner is built with
   USE_PFRING_ZC=1, dispatch finally reaches pfring_zc_sender_thread /
   pfring_zc_receiver_thread (it never did before — the ZC code paths were
   compiled but unreachable due to the hardcoded dispatch).
5. AnyScan-side script updates so install-external-deps.sh and
   package-worker-bundle.sh resolve the scanner from the fork.

The scanner-fork commit is at:

• Repo: https://github.com/AnyVM-Tech/anyscan-engine-c
• Branch: perf/portscan-afxdp-phase2-pr1
• Commit: 998c66b
• PR (companion):
  AnyVM-Tech/anyscan-engine-c@main...perf/portscan-afxdp-phase2-pr1
• Diff stat: 7 files changed, 374 insertions, 14 deletions.

Why a fork (per plan §9.1)

The plan documented three options for where AF_XDP changes should live:
upstream PR, AnyVM-Tech fork, or a patch carried in
install-external-deps.sh. Recommendation in the plan was fork at
AnyVM-Tech/anyscan-engine-c, and that is what this PR adopts. The fork is
the canonical location going forward for:

• The AF_XDP integration patches (PR 2 + 3 of Phase 2).
• The Makefile / build integration for USE_AF_XDP (PR 4 of Phase 2).
• The follow-on PF_RING ZC cluster initialization (separate from this PR).
• Any divergence we accumulate from upstream.

upstream (Lorikazzzz/VulnScanner-zmap-alternative-) is preserved as a
remote on the fork so we can rebase against future upstream commits if/when
that becomes useful.

Dispatch refactor — the change in pictures

Before (src/engine.c:165, hardcoded):

scan_ctx[i].socket_fd = socket(PF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
struct sockaddr_ll sll = { .sll_family = AF_PACKET, ... };
bind(scan_ctx[i].socket_fd, ...);
pthread_create(&senders[i], NULL, sender_thread, &scan_ctx[i]);

(PF_RING ZC sender exists in src/send-pfring.c but is never invoked.)

After:

const io_engine_vtable_t *io = pick_io_engine(config->io_engine);
if (!io) exit(1);
...
io->init_per_thread(&scan_ctx[i], config);
pthread_create(&senders[i], NULL, io->tx_thread, &scan_ctx[i]);
...
pthread_create(&receivers[i], NULL, io->rx_thread, &r_ctx[i]);

The vtable shape mirrors plan §3.3:

typedef struct {
    const char *name;
    int   (*init_per_thread)(thread_context_t *ctx, scanner_config_t *config);
    void *(*tx_thread)(void *arg);
    void *(*rx_thread)(void *arg);
    void  (*teardown_per_thread)(thread_context_t *ctx);
} io_engine_vtable_t;

CLI behaviour

Build flags	--io-engine=af_packet	--io-engine=pfring_zc	--io-engine=af_xdp
(default)	runs (default)	exits 1 — "USE_PFRING_ZC=1 not set"	exits 1 — "USE_AF_XDP=1 not set; lands in PR 2 + 3"
USE_PFRING_ZC=1	runs	dispatches into the ZC threads (was previously unreachable)	exits 1 — same as above

Unknown values (e.g. --io-engine=bogus) exit 1 with a clear "Unknown
--io-engine" error before any setup runs. All errors land at parse time, so
none of these tests need CAP_NET_RAW.

Files in this PR (AnyScan side)

• install-external-deps.sh — default URL and local-checkout dir now point
  at the AnyVM-Tech fork (anyscan-engine-c). Both stay overridable via
  ANYSCAN_VULNSCANNER_REPO_URL / ANYSCAN_VULNSCANNER_REPO_DIR.
• package-worker-bundle.sh:519-525 — bundle lookup order is now
  anyscan-engine-c/scanner first, then the legacy
  VulnScanner-zmap-alternative-/scanner (transitional, for dev checkouts
  that still have the old directory name), then /opt/anyscan/bin/scanner.

Files in the companion fork commit

• include/scanner_defs.h — IO_ENGINE_* enum and int io_engine field.
• include/scanner.h — io_engine_vtable_t, pick_io_engine(),
  io_engine_from_string(), io_engine_name() declarations.
• src/conf.c — --io-engine parsing, compile-flag gates, help text.
• src/engine.c — vtable definitions for AF_PACKET (and PF_RING ZC under
  #ifdef USE_PFRING_ZC); dispatch refactor in run_scan. ICMP prescan
  helpers continue to use their own raw PF_PACKET sockets independently of
  the chosen TX engine.
• Makefile — make test target.
• tests/io_engine_dispatch.sh — 11 smoke tests covering the CLI + dispatch
  matrix above.

Test plan

AnyScan repo (this PR):

• cargo build --workspace --release — clean (1m04s, 2 pre-existing
  dead_code warnings only).
• cargo test --workspace --no-fail-fast — 437 tests pass, matches
  the post-perf(portscan): AIMD CPU-vs-network slip + subprocess cap + per-instance defaults + partial-window calibration #66 baseline (371 + 31 + 2 + 33, plus 0 doc-tests, with 4
  pre-existing ignored).
• python3 -m py_compile vulnscanner-zmap-adapter.py — clean.

Scanner fork (AnyVM-Tech/anyscan-engine-c):

• make (default AF_PACKET build) — builds clean (pre-existing
  strncpy warnings unchanged).
• make test — 11/11 dispatch smoke tests pass.
• gcc -fsyntax-only -DUSE_PFRING_ZC -I<stub> -Iinclude src/engine.c src/conf.c src/send-pfring.c src/recv-pfring.c — clean. (The
  environment here doesn't have libpfring-dev to do a full link, but the
  syntax check confirms the dispatch wiring compiles under
  USE_PFRING_ZC=1; the dispatch fix is a one-line indirection that
  reviewers can verify by inspection.)
• ./scanner --io-engine=af_xdp -p 80 exits 1 with the expected
  "USE_AF_XDP=1 not set" message.
• ./scanner --io-engine=pfring_zc -p 80 exits 1 with the expected
  "USE_PFRING_ZC=1 not set" message (when not built with that flag).
• ./scanner --io-engine=bogus -p 80 exits 1 with the "Unknown
  --io-engine" error.
• ./scanner --io-engine=af_packet --help works.

Backwards compatibility

• The default behaviour is unchanged. Without --io-engine, the binary runs
  the same AF_PACKET path it always did. The inline socket(PF_PACKET, ...)
  • bind(...) from the previous run_scan is moved into
    af_packet_init_per_thread so the dispatch can stay symmetric across
    engines. (Side effect: bind()'s return value is now checked, where
    previously it was ignored — a small robustness improvement.)
• Worker bundles built before this PR continue to work; the fork URL/dir
  defaults only affect new clones via install-external-deps.sh.
• The legacy VulnScanner-zmap-alternative-/scanner lookup path is kept in
  package-worker-bundle.sh as a transitional fallback for dev checkouts
  that still have the old directory.

What is NOT in this PR

These ship in subsequent Phase 2 PRs:

• PR 2 — src/send-afxdp.c (XSK setup, UMEM alloc, TX ring batching,
  completion-ring drain, kick path) per plan §3.4.
• PR 3 — src/recv-afxdp.c + the matching io_engine_af_xdp vtable
  registration on the fork.
• PR 4 — Makefile USE_AF_XDP=1 block, install-external-deps.sh
  build-flag plumbing, libxdp-dev / libbpf-dev dependency lines, ENA
  channel-half / SKB-mode fallback runtime probes, and systemd-unit
  CAP_BPF ambient capability per plan §3.5–§4.4.
• PR 5 — live c6in.metal multi-NIC bench, AIMD-controller coordination
  with anygpt-33.

Out-of-scope for this and all subsequent worker tasks (per plan §9 and
coordination notes):

• anyscan_rate_controller.py and the AIMD adapter — owned by anygpt-33
  (just merged in perf(portscan): AIMD CPU-vs-network slip + subprocess cap + per-instance defaults + partial-window calibration #66, kept untouched here).
• /etc/anyscan/runtime.env on prod — owned by ops.
• AnyGPT submodule pointer bump.

Refs

• Plan PR: docs(plans): AF_XDP integration plan for higher pps (Phase 1) #65 (docs(plans): AF_XDP integration plan for higher pps (Phase 1)).
• Plan file: plans/2026-04-27-portscan-afxdp-plan-v1.md §3.1, §3.3, §9.1.
• Pre-existing AIMD/perf-portscan work this PR is independent of: perf(portscan): AIMD CPU-vs-network slip + subprocess cap + per-instance defaults + partial-window calibration #66.
• The PF_RING ZC dispatch wart this PR also fixes:
  Lorikazzzz/VulnScanner-zmap-alternative-'s src/engine.c:165,
  src/send-pfring.c:4, src/recv-pfring.c:4.

🤖 Generated with Claude Code