Skip to content

v0.3.1 — the tools audit themselves

Choose a tag to compare

@PranavNagrecha PranavNagrecha released this 22 Aug 14:53
· 79 commits to main since this release

A correctness release against tools that were already shipped. No new capability tier.

Three adversarial audits went through the permissions, schema-and-search, and automation tool families, hunting only tools that predated the work. They produced 39 verified defects — each reproduced against a real org vault, measured for scale, and checked against a sibling tool answering the same question. Then a review pass and an adversarial QA pass found 18 more that the fixes themselves introduced, all of them invisible to a green suite.

Both numbers are reported here on purpose. A release note claiming a clean sweep, from the same work that shipped the original 39, would not be credible.

What moved

Measurement Before After
Apex classes reported likely dead, of 186 85 0
find_dead_code likely_dead (both vaults) 18 2
definitely_dead at org scale (the control) 1041 / 979 unchanged
Classes reported as having no test 58 19
Save-order steps returned for a busy object 5 of 57, marked complete 57 of 57
Share of a save response spent on the answer 12% 78%
Rows reachable by paging a code-quality audit 94 of 647 647
Rows reachable by paging an unused-field audit 281 of 570 570
Trigger-deactivation verdicts carrying information 0 of 22 22 of 22
"What runs when I save a contact?" answered end to end no yes

Four worth naming

A verdict justified by something untrue about Salesforce. find_dead_code told you it was safe to delete classes that run on a schedule, reasoning that such a class "must be enqueued/executed/scheduled by user Apex." An admin scheduling through Setup creates a CronTrigger record — data, not metadata — never retrieved, no node, no edge, and no refresh can ever close that gap. 16 of 18 likely_dead verdicts on each vault were exactly those classes. They are now uncertain; definitely_dead is unchanged, so the control held.

A stale retrieval overwriting the current one. A vault can hold two complete retrievals. Node writes replace on conflict and the directory walk is alphabetical, so the older copy won. On a real org, three profiles reported an MFA-bypass permission the current retrieval no longer declared. The canonical layout now wins and every conflicted component carries a sourceConflict field naming both paths and how the answer was chosen — or says precedence: undetermined rather than guessing.

The honesty payload was being truncated. The response trimmer learned to descend a level, which made the disclosure arrays reachable — the lists of what a tool did not check — and began shortening them under a note claiming a total that was never published. Disclosures are now never trim candidates; an oversize response refuses and names real narrowing knobs instead.

Resume pointers that skipped whole windows. A pointer computed from what a handler intended to return, rather than what survived a second trim, made rows unreachable by any call. Pointers are now corrected only on a positive match against a page size the payload itself publishes, and invalidated with a stated reason otherwise.

What was actually wrong, underneath

Almost every defect was a second copy of something — two same-named constants, a block duplicated under a comment promising byte-identity, a hand-copied JSON Schema beside the validator that enforces, two rule counters, two byte limits that had to stay ordered and were set independently in the wrong order. Every one written by someone who knew. The comment was the mechanism, and comments do not hold.

So the durable part of this release is not the fixes:

  • an advertised-vs-enforced schema parity gate over all 217 tools on four axes, with 27 pre-existing violations baselined and reasoned rather than hidden
  • a tool-local byte budget derived from the global one, so the ordering holds by construction
  • drift tests running two implementations of a shared predicate against one fixture
  • a documentation pin that could not fire — its pattern could not match across a line break, so a stale count sat beside three correct ones and the file reported clean

Upgrading

No tool's input contract breaks an existing call. Ten tools gained advertised inputs they already accepted, one dropped an input that was a pure synonym for the canonical one, and several now refuse selector combinations that previously resolved silently to something you did not ask for.

If your vault holds more than one retrieval of the same org — a source tree with both a flat and an SFDX layout — rebuild it. Until you do, the tools will tell you which components are affected instead of quietly answering from whichever copy sorted last.


11,412 tests green. Full detail for this release is in CHANGELOG.md — the complete section runs to ~140,000 characters, past GitHub's release-note limit, so this page is a summary.

Release post: https://sfi.auditforce.cloud/blog/sf-intelligence-0-3-1