-
Notifications
You must be signed in to change notification settings - Fork 0
Lessons
Mistakes that were actually made here, and the rule each one produced. Read this before repeating one someone already paid for.
The headline finding, after auditing a full day of them: seven of eight errors were verification gaps, not knowledge gaps. The fix is almost never "know more". It is "check before asserting".
A checksum verify is not an integrity check. Three PBOs with a leading \ on
every entry path verified clean and took the server down for six hours. Read the
entry table. → Runbook §6
Blaming the wrong component costs the most time. "pbo.ps1 is not at fault" was asserted, then disproved by reproducing the bug from a short-name path. The tool was the root cause. Reproduce before exonerating.
Symptoms outnumber causes in a loop. extDB2 is already setup & locked and
Unknown Protocol were treated as database problems for hours. Count
Starting mission: first.
Measurements taken at the wrong moment are lies. An object count of 269 was reported as the plateau; it was taken mid-population and the real figure was 576. The improvement was ~17%, not 61%. Let the system settle before measuring.
"FPS decays with uptime" was wrong. It was a lootbox deadlock —
LB_WaitSysBusy at 40 stalled population at 66/103. Fixing it took FPS from 22
to 46 and completed the map. A plausible-sounding pattern is not a diagnosis.
Do not dismiss volume as noise. 160 Sign_sphere100cm_EP1 errors were
written off as harmless. It is an Arma 2 class — the inner safezone ring was
invisible. Corrected to Sign_Sphere100cm_F, errors went to zero.
Read the enclosing block before calling something a bug. SC_mapMarkers = true
was flagged as leaking loot positions. It sits inside if (SC_debug), which is
false.
PowerShell .Replace() with \r\n on an LF file does nothing — and returns
success. Two INI edits "succeeded" and changed nothing, then a meaningless test
"verified" them. Use awk, and verify by printing the section back.
Set-Content -Encoding utf8 writes a BOM. A BOM in xcsv_guard.json made
serde_json reject the file, so the app silently ran on Default — no paths, no
credentials, and no error. The parser now strips it, with tests.
A Rust module that is never declared never compiles. mod secrets; was
missing from main.rs, so credentials stayed plaintext while the documentation
said otherwise. Adding a file is not adding a feature.
Nested regex through bash quoting produces confident garbage. It reported "288 corrupt entries" in a perfectly healthy PBO. Use the dedicated search tool, or write the pattern to a file.
Credentials live in more places than you remember. After an RCon password
rotation the console could not authenticate — the old value was still in
xcsv_guard.json. Five locations had to be updated. Grep for the old value
before declaring a rotation done.
Seven failed hypotheses on the headless client — BattlEye, SteamCMD binary,
loopback vs LAN, missing slot, server busy, @A3XAI_HC, startup race — produced
nothing. A packet capture produced the only hard fact in the whole investigation:
1107 packets server→HC, all dropped; 0 packets HC→server. When hypotheses stop
converging, stop hypothesising and measure.
A relaxed check is sometimes the correct fix. The PBO checker judged a
missing prefix header as Broken and blocked autostart. Three live addons ship
that way legitimately, deriving their path from CfgPatches. It is a warning,
not an error. Confirm against reality before hard-failing.
A latch that is never cleared runs once. stack_probing was set and never
reset, so the probe fired a single time and then appeared to work forever.
Anything set by one thread should be cleared by the one that finishes the job.
- Verify, then assert. Not the other way round.
- Reproduce before exonerating a component.
- Read the enclosing block before calling something a bug.
- After any file edit, print the result back — especially through PowerShell.
- Correct the record when wrong, in the same turn, plainly.
- Record dead ends, not only successes. Seven failed hypotheses are worth writing down precisely so nobody spends the evening on hypothesis eight.