perf: add full vs incremental flist memory benchmark (#1864)

[oferchen]

Summary

• Adds scripts/benchmark_flist_memory.sh measuring peak RSS of oc-rsync vs upstream rsync 3.4.1 at 100K and 1M directory scales under three flist modes (full / receiver INC_RECURSE / sender INC_RECURSE-pending).
• Documents 2026-05-01 baseline at docs/benchmarks/flist-memory-baseline-2026-05-01.md.
• Refs Improve rpm packaging error reporting #1864 (this benchmark), Route CLI fallback through core orchestration #966 / Enable environment-driven system rsync delegation in oc-rsyncd #971 (RSS gap context). Mode C (sender INC_RECURSE) is skipped pending Make Debian alternatives opt-in and clarify amd64 tarball distribution #1862 opt-in flag.

Modes

Mode	Description
A	Full flist (--no-inc-recursive)
B	Default (receiver INC_RECURSE; sender always sends full list)
C	Sender INC_RECURSE (skipped - awaiting #1862 opt-in flag)

Container invocation

podman exec rsync-profile bash /workspace/scripts/benchmark_flist_memory.sh \
    --scales both --summary

Fixtures live under /tmp/oc-rsync-bench inside the container (not bind-mounted). Cleanup uses a path guard that refuses any rm -rf outside that prefix to avoid the bind-mount accident class. TSV/MD output is written to target/benchmarks/.

Helper flags: --scales 100k|1m|both, --summary (markdown table), RUNS=N env (default 3).

Baseline numbers (oc-rsync v0.5.9 vs upstream 3.4.1, aarch64-linux)

100K files

Mode	Binary	Wall (s)	Peak RSS (MB)
A	oc-rsync	2.81	42.7
B	oc-rsync	2.91	42.6
A	upstream	3.68	14.2
B	upstream	4.39	7.9

1M files

Mode	Binary	Wall (s)	Peak RSS (MB)
A	oc-rsync	53.6	218.2
B	oc-rsync	53.6	218.5
A	upstream	49.0	76.8
B	upstream	58.7	7.5

Key findings

• Upstream INC_RECURSE delivers a 10x RSS reduction at 1M (76.8 -> 7.5 MB).
• oc-rsync shows no Mode A vs Mode B difference because sender-side INC_RECURSE (Make Debian alternatives opt-in and clarify amd64 tarball distribution #1862) is not yet active. Local push always exercises the sender role on the parent process.
• RSS gap: 5.4x at 100K, 29x at 1M. The gap widens linearly with file count and motivates the Make Debian alternatives opt-in and clarify amd64 tarball distribution #1862 work.

Reproduction cost

Scale	Generation	Per push (median)	Total wall
100K	3 s	~3 s	~50 s
1M	41 s	~50-58 s	~12 min

Test plan

• bash -n syntax check on the script
• 100K run completed end-to-end inside rsync-profile container; TSV+MD written
• 1M run completed end-to-end inside rsync-profile container; TSV+MD written
• Path guard rejects paths outside /tmp/* or /var/tmp/* prefix
• Script handles missing oc-rsync binary, picks oc-rsync-dev when present
• Re-run after Make Debian alternatives opt-in and clarify amd64 tarball distribution #1862 lands to populate Mode C numbers

Notes

• The benchmark is not wired into CI; it is a developer/release tool intended to track flist memory progress over time. No new dependencies are added.
• Working-tree mode-bit churn (644 vs 755) on the script file may show up locally; the index has it 755 so the committed/pushed version is executable.