Skip to content

v0.3.3

Choose a tag to compare

@github-actions github-actions released this 03 Oct 20:58
· 8 commits to main since this release

Added

  • OmniStudio is modeled from the inside. Every OmniScript, Integration Procedure, FlexCard and DataMapper is parsed into its elements through one element catalog (canonical types, every alternative spelling, runtime roles — generated into docs/omnistudio/element-catalog.md). New edges: any element naming a remoteClass calls that Apex class (callsApex, with methods and call sites — Apex called only from OmniStudio no longer looks dead); DataMappers read and write individual fields (Extract, Turbo, Load — a Load's object write says create or upsert); an Integration Procedure Delete Action deletes the objects it names; a FlexCard SOQL data source reads its object and fields. Cross-references now land on the version that runs (an IP's isActive is the runtime switch; a DataMapper's active flag is not), so impact and dependency walks cross from a screen into its server code.
  • Ten OmniStudio tools: omni_model, omni_save_trace (does each answer reach a field?), omni_prefill_trace (does it come back on reopen?), omni_dead_references, omni_version_diff, omni_edit_block_audit, omni_completion_audit, omni_form_spec, omni_path_simulator (now with DEAD_END_SECTION: answers that add a section while its entry step hides itself, found by a satisfiability search over the step's rule expanded through the script's formulas) and omni_changed_since (semantic diff of what changed since a snapshot).
  • Four audit tools: record_delete_impact (cascade / orphan / restrict per lookup, delete guards, who can delete — including generic Apex delete helpers resolved at their call sites), persona_unused_grants (object and field permissions nothing a persona can run uses; never UNUSED when it cannot tell), live_verify (re-check one vault claim against the org, read-only and consent-gated) and how_to_see (the Setup click path to see a component).
  • sfi run <tool> --json --<arg> <value> … --vault <path> calls any tool from a script through the MCP dispatch path; arguments are typed from the tool's own schema.
  • --vault <path> means one thing everywhere — run, status, doctor, selftest, snapshot, refresh, feedback export, gaps report, review-change — and accepts the org-kb folder or the folder that holds it. gaps report --vault counts only that vault's gaps.
  • App scope for counting tools. picklist_integrity_scan and doc_coverage_report accept scope {namePrefixes, namespaces} (default: org-kb/config/app-scope.json, else the OmniStudio appScope) and answer with the scoped number, the org-wide number beside it as a labelled contrast. picklist_integrity_scan now reports how every picklist is restricted (global value set / inline restricted / inline unrestricted / unknown).
  • The field extractor records a picklist's restricted flag and a lookup's deleteConstraint; snapshots record runtime switches and a hash of each OmniStudio source file.
  • Managed-package (Vlocity) OmniStudio. A Vlocity Build Tool DataPack export placed under org-kb/source/vlocity/ (OmniScript, IntegrationProcedure, DataRaptor and VlocityCard DataPacks, with their sibling files; the %vlocity_namespace% placeholder or a real vlocity_cmt / vlocity_ins / vlocity_ps / omnistudio prefix) is converted to the native component's shape and extracted by the same code as native metadata — same node ids, edges, graph resolution, and every OmniStudio tool. Nodes carry sourceFormat: vlocity-datapack and cite the DataPack file; a snapshot fingerprints the whole DataPack folder, so omni_changed_since sees an edit to a sibling file; other DataPack types are disclosed as skipped, and the refresh never deletes an export. OmniStudio tools say in trust.limitations when an installed Vlocity package has no export (its components are missing, not absent) and how many components came from one. Export recipe in docs/configuration.md.
  • Apex that runs OmniStudio is a caller in the graph. <ns>.IntegrationProcedureService.runIntegrationService('Type_SubType', …) and <ns>.DRGlobal.process(…, 'Bundle') / processObjectsJSON with a literal key (namespace omnistudio, vlocity_cmt, vlocity_ins or vlocity_ps) become dispatchesOmniAction edges from the Apex class (via: apex), resolved onto the version that runs — so usage, impact and unused-component answers count Apex callers, and an IP called only from Apex no longer looks unused. The call no longer mints a phantom ApexClass:IntegrationProcedureService / DRGlobal or a CustomField:<ns>.<Class> read. A key built at runtime is not guessed. New concept apex-runs-omnistudio.
  • The managed runtime's IntegrationProcedureService is an IP call, not Apex. A Remote Action on <ns>.IntegrationProcedureService dispatches to the Integration Procedure its method names (dispatchesOmniAction, via: runtime-service); the save and prefill traces follow it into that IP, and integration_procedure_chain lists it as an integration-procedure endpoint.
  • Concepts and rules for the new semantics (DataMapper writes, remote Apex calls, Edit Block delete wiring, IP Delete Actions, lookup delete behavior, picklist restriction, dead-end sections, DataMapper active not being a runtime switch, managed-package OmniStudio records).

Changed

  • integration_procedure_chain walks steps nested in blocks, classifies every spelling through the catalog, names the active version of a multi-version target (others listed), resolves Apex calls to ApexClass nodes, reports Delete Actions, and sums what the chain reads, writes and deletes. Its and omniscript_flow's Apex disclosure now says what is and is not modeled.
  • The OmniStudio formula evaluator computes CONTAINS and returns computed values, not only truth.
  • The Integration Procedure extractor compares element types through the catalog, so an HTTP Action callout is recorded like a Rest Action.
  • An Apex class OmniStudio calls by name that the vault does not hold says what that means and how to tell a missing class from a package service.
  • The five OmniStudio component tools share ONE native-vs-managed-package disclosure, which now says where managed-package components come from instead of that they are not extracted.
  • Every tool the release added is registered in the high-fan-out inventory with a real-org probe (the 0.3.3 registration backlog is empty).
  • Question routing keeps the older tools reachable next to the new OmniStudio vocabulary: "which OmniScripts exist" reaches the inventory tool (list_components, through a component-family card that dilutes nothing else), and "would the platform block deleting this field" still reaches safe_to_delete_field.
  • The default tool profile hid the tools the product's own front page promises it can answer. The package documentation invites readers to ask "what breaks if I delete this field?", "who can edit this field?", "why can't this profile see this record?", "what happens when I save this?", and "give me a tour of this org" — and the tools that actually answer those questions were among the 198 of 217 tools the default profile did not advertise. A schema-driven MCP host reads the tool list once per connection and will not invent an internal gateway tool on its own, so the product answered its own advertised questions only for a caller who already knew that gateway existed. The token-cost argument for a small default roster was real but had never been measured: measured now, the six newly-added tools cost about 36.5 KB against a 665 KB full roster and a 67 KB spine — the default grows from 67 KB to 104 KB, about 16% of the full roster, specifically for the tools the front page promises. The default ("core") profile is now derived, not hand-picked, from a map pairing each published question to the tools that answer it, checked by a test that every question in that map appears verbatim in the package README — so a promised question with no tool behind it now fails the build instead of failing a user. Default tool count: 19 → 25.
  • Twenty-two tools, including the single most-called tool in the roster, advertised every argument as optional and then refused to be called with none at all. Each takes a selector under multiple accepted spellings (an id under two different names, an object under three), so the real contract is "at least one of these," and nothing in the advertised schema said so — a schema-driven host had no way to know an empty call would be refused rather than accepted. The existing schema-accuracy check measured required arguments by removing one key at a time from a full example input; with two valid spellings present, removing either one alone still parses, so the check had structurally never asked the one question that would have caught this: can this be called with no arguments at all? A new shared helper, applied uniformly across all twenty-two tools rather than as twenty-two hand-written variants, closes the gap.
  • Seventeen tool descriptions were rewritten to match the object-scope-refusal fixes above — the description is the contract a host LLM reads before deciding whether to call a tool, so a stale one is not cosmetic. One had gone further than stale: it had documented its own defect as a deliberate feature, describing an unmodeled object's empty result as clean rather than unchecked. A new drift ratchet now tracks hand-rolled "was this actually extracted" checks that should route through the product's shared helper for that instead, so the pattern this release fixed repeatedly cannot silently regrow.
  • The roster-count change rippled outward: four tests that had pinned the old count of 19 as a literal now derive it from the same source of truth, and six more places — across the contributor guide, the configuration docs, and both website pages — that still described the default profile as "19 tools" or as merely "the spine" (rather than "the spine plus the tools the package page advertises") were corrected and are now checked by the project's documentation-sync gate.
  • The Windows CI job runs as a real hard gate again, and it stayed that way this time. A prior commit on this same release line had added a hard-gate tier and then, one commit later, silently walked it back to advisory again with no explanation. The evidence: a run on main reported windows-build-test SUCCESS over a log that read packages/mcp test: Failed / Exit status 3221226505 (0xC0000409, a native Windows crash) with no test summary printed at all — the majority of the tool surface had never actually finished executing on Windows, and the job was green anyway because both test steps carried continue-on-error: true. The seven packages with no DuckDB dependency (core, tooling-api, renderers, parsers, patterns, vault, extractors — 130 files) already passed 100% on windows-latest and cost nothing to gate for real; they now run hard, with no continue-on-error, first after checkout. A new pnpm check:portability step also moved onto Windows itself rather than asserting POSIX-only guarantees from Linux. Switching the test runner to pool: 'forks' was tried and measured rather than guessed — 88.16s (threads) vs 156.37s (forks) on a matched macOS pair, ~77% slower for no observed correctness difference — and reverted. A new anti-backslide test (test/ci-workflow-invariants.test.ts) now parses ci.yml itself and asserts the hard tier only ever grows, carries no continue-on-error, and that every remaining advisory step names a tracked blocker — proven to catch both silently re-disabling the hard tier and silently dropping a package from it.
  • Found the actual cause of the Windows crash: a code comment that had been false since the day it was written. packages/mcp/vitest.config.ts carried a comment, copied from a sibling package's real fix, asserting its test timeout and hook timeout were "EQUAL BUDGETS; a hook that truly hangs still fails" — the two numbers underneath it were 45000 and 20000. The setup hook that opens the DuckDB-backed graph for the package's heaviest test fixture (401 nodes, 400 edges) ran on the smaller, unraised clock while the tests that merely query it ran on the larger one; when vitest kills a hook mid-DuckDB-call, the native layer aborts the whole process (the same abort as 0xC0000409 on Windows and 134 on POSIX, two encodings of one crash) instead of failing softly — which is why the job printed nothing rather than a partial summary. The fixture completes in 985ms locally and only exceeds its 20-second hook budget under thread-pool contention on a loaded CI runner, which is why it never reproduced locally and why two earlier attempts — raising the test timeout, and trying pool: 'forks' — both missed it. Both clocks now derive from one constant per package instead of being asserted equal in a comment that can drift. A new gate, test/vitest-timeout-parity.test.ts, found five more instances of the same asymmetry on its first armed run: three packages set a test timeout and never set a hook timeout at all, silently sitting on vitest's 10-second default, and two more wrote the same pair of numbers twice in two places, which agrees only until one copy is edited. Not verified on an actual Windows runner — none is available locally, and this is stated rather than assumed: the fix removes the starvation, it does not make a killed DuckDB call fail softly, and if the abort recurs the documented next step is isolating the DuckDB-backed suites so one crash cannot erase hundreds of other files' results.
  • A green Windows tick had stopped meaning "verified," and nothing said so. The release gate that decides whether to publish to npm trusts the CI run for the tagged commit concluding successfully, and is written fail-closed everywhere except here: it trusted the Windows job's color without checking what the color actually covered — which, on the prior release, was a job whose core test suite had aborted with no summary at all. The Windows job now reads each step's real outcome (the result from before continue-on-error is applied) and publishes a machine-readable report of what each tier actually did; the publish workflow downloads and checks that report rather than the job's overall color, and refuses on a missing, empty, or hard-tier-not-successful report, including the case where the run itself shows green. A red advisory tier still allows a release — blocking every release on a Windows crash nobody can reproduce off-Windows would be its own dishonesty — but partial Windows coverage is now stamped into the GitHub release body itself, not just a warning line in a log nobody reads afterward. A new repository setting can turn that partial-coverage case into a hard refusal; it is unset by default, so release behavior is unchanged until a maintainer opts in.

Fixed

  • A --with-tooling-api refresh that could not capture permission dependencies said so nowhere. A sign-in failure stopped the pass before the capture without naming it, an enrichment error hid the capture line, and --staged dropped the flag. The summary now prints a WARNING naming the cause on every path, and the refresh ends partial.
  • A copied vault could refresh the vault it was copied from. A config whose vaultRoot names another folder is now refused by every command that writes to the vault, with the exact fix.
  • OmniStudio cross-references no longer dangle onto phantom stubs with a "retrieve it" remedy; DataMapper input paths no longer mint phantom objects; an inactive DataMapper is no longer treated as dead; Edit Block deleteIPKey / saveIPKey are read.
  • A remoteClass in a namespace containing an underscore (vlocity_cmt.ClassName) was dropped; it now links to its vlocity_cmt__ClassName component.
  • Consecutive offline refreshes are idempotent for Report / Dashboard coverage. A refresh that pulled no reports read the previous manifest's capped-retrieve evidence but did not write it on, so the next --no-pull refresh — or the final tier of a staged build — reported Report / Dashboard as pending where a single refresh of the same source reported them captured.
  • A key dropped at a DataMapper Load now names, in its near-miss suggestion, the field that item would have written (it was always empty; a Transform's already did).
  • "Pull up every formula that references Amount__c" / "which formulas are referencing X" routed nowhere: the formula-reference rule matched only the bare verb. They now route to find_formula_references.
  • Routing honesty. "Run automation risk for regression" / "execute the flow audit" were refused as requests to EXECUTE automation — a run verb on an analysis name is now a read (a run of the flow itself is still refused). Out-of-scope asks that slipped into low-confidence tool routes now get the honest out-of-scope answer: another platform's own data ("how many SAP tables"), rating people, purchasing decisions, general knowledge. A funnel-only (advisory) route now also requires the product to know a third of the user's words, so a gibberish question carrying one Salesforce word stays unrouted. "Resolve the component named X" routes to the resolver.
  • No phantom class from Schema describe chains. Schema.SObjectType.Account.getRecordTypeInfosByName() (and ….fields.Name.getDescribe()) minted a callsApex ApexClass:Account / ApexClass:Name edge in every org using the idiom; the token in a describe chain is an object or field name, never a class.
  • "Which orgs in our fleet have drifted most from the baseline?" reaches fleet_drift_ranking first again (it was ranked under fleet_find), and "what flows can the guest user run" keeps user_ability in the shortlist.
  • what_if_deactivate_flow did not know a Flow embedded in a Lightning page is a live runtime dependency. Measured on the real org: 12 edges representing a Flow embedded directly in a Lightning record page via a Flow component, every one silently dropped from the impact analysis. A Flow placed on a page this way certified a "safe" verdict with no entry points and no limitations — "nothing depends on this" — while any user opening that page after deactivation would hit a runtime error in that component's region. The tool had excluded this kind of edge on the theory that a page embed is "access, not usage," borrowed from a discipline that applies to a different tool checking who merely has visibility into a component — but an embedded Flow component actively invokes the Flow at render time; it is not a passive grant. It is now graded as a blocking dependency unconditionally, since a page embed carries no active/draft signal the way a parent Flow does, and there is no basis to soften the verdict.
  • find_dead_code trusted a precomputed edge mirror that had never actually run against this vault. Some Salesforce metadata families let the same physical custom field appear in more than one form (for example, both a Task-shaped and an Event-shaped representation of one field), linked by an internal mechanism that is supposed to mirror usage evidence across those forms. Measured on the real org: a field with real usage on one of its representations — seven Flow-decision reads, a validation rule, a list view, two layout placements — had its sibling representation, the same underlying field, reported definitely dead with zero incoming edges. The mirror that should have linked them had never actually run on this vault at all (not one edge of that kind exists anywhere in it), most likely because the vault predates the mechanism or an incremental refresh under-mints it. The fix runs at query time instead of depending on the mirror: before certifying a field in one of these families definitely dead, the tool now cross-checks its other representations' own real usage edges directly, and only confirms the verdict when none of them show any usage either.
  • Three different tools gave three different answers about the same empty metadata family on the same vault, in the same run. Measured: a metadata family with zero retrieved components, and 79 real edges from other components naming 31 distinct members of it that were never actually retrieved. In one session, on that vault: one tool correctly said "not a checked zero"; a second said "none in the org"; a third said the family was fully covered. Three tools, one underlying fact, three contradictory stories — the disagreement itself was the defect, independent of any single tool's own logic being locally correct. The right check already existed inside the first tool; it is now shared, so the second tool reports "not checked" with the real edge and member counts instead of a false "none," and the third downgrades a graph-contradicted zero from "covered" to "partial."
  • unused_fields_deep had the identical blind spot find_dead_code had just been fixed for — found independently, on the same vault. 292 of this vault's 4,296 custom fields sit on the family described above, and 18 real field groups have one representation with genuine usage and a sibling with none. This tool certified the zero-evidence sibling as high-confidence "consider deletion" across all eight of its scoring checks, for the exact field find_dead_code had already learned to call uncertain rather than dead. The remaining gap was narrower than the fix above: five of the eight checks already match by field name across the whole vault and so already caught a sibling's usage, but the incoming-edge check scores per graph node, and an edge landing on only one of up to three representations of the same physical field stays invisible to an otherwise-clean sibling. The same cross-check added to find_dead_code now runs here too, as a final pass after the eight-check scoring: a high-confidence survivor from one of these families is downgraded to medium confidence when another of its representations shows real usage this one does not. This was the fourth and last critical defect from the real-org persona testing pass.
  • Four org-wide inventories reported an alphabetical first page as a complete org. org_overview ranked "top objects by inbound reference" and "top Apex classes" over the alphabetically-first 500 nodes and computed automationSummary.activeRatio as a 500-node numerator against a COUNT(*) denominator; integration_map divided its row budget eight ways and returned ~13 rows per category in an output type with no truncation field at all; endpoint_catalog advertises "every URL / endpoint" and reported summary.totalEndpoints from a 500-per-category scan; test_coverage_gaps built both its class corpus and its test-class set from one 500-row page, so a test class sorting past position 500 was invisible and its callers were reported uncovered. All five tools (including generate_admin_handbook) now scan to exhaustion through the existing scanAllNodesOfTypes helper, and full-scan-adoption.test.ts fails if a single-page corpus scan reappears in any of them.
  • test_coverage_gaps returned gaps: [], totalGapsCount: 0 for a classFilter naming ids that are not in the vault — an unchecked zero indistinguishable from a clean bill of health. Unknown ids are now named in notFoundClassIds and disclosed in boundaries[].
  • compare_profile_across_vaults reported "0 object / field / Apex permission drift" from three node properties the Profile extractor never writes. It read properties.objectPermissions / fieldPermissions / apexClassAccesses — the extractor writes counts, not grants — so the tool fabricated a confident measured zero. Grants are now read from grantedBy edges, as compare-components.ts already did. extractGrantMap also dropped every userPermissions entry to a typeof entry === 'object' guard against what is really a string[], reporting 8 real differences as 0. The now-populated arrays are paged through the shared paginateSection cursor with the byte budget derived from toolLocalPayloadBudgetBytes(), so the tail is reachable rather than silently trimmed.
  • why_cant_user_see_record returned a confident restricted hard deny for a profile or permission set that is not in the vault at all — a deny computed from a container nobody read, stated as a measured fact. A caller-supplied container that cannot be read now yields unknown with the gap named. PermissionSetGroup MEMBER ids are deliberately excluded from that rule: they are synthesized from bare names and are routinely unextracted, so those misses are disclosed on the PermissionSetGroup step and the verdict still resolves.
  • sfi refresh's side-build swap could throw a raw errno out of the CLI. Every filesystem call in the swap is now inside the catch, the drifted inline lock-message regex is replaced by the classification GraphError.kind already carries, and the failure message states what actually happened: the database was not replaced, answers are STALE not broken, whether the scratch rebuild was really cleaned up, and that graph/resolve-index.json beside the database was overwritten from the discarded build and should be deleted. Not fixed, and stated rather than papered over: on Windows a refresh still cannot swap a vault database while a client holds it open — that needs generation-named database files, which is a larger design than this release.
  • sfi vault anonymize's containment rails compared paths case-sensitively, so on the case-insensitive filesystems this ships on (NTFS, default APFS) an --out spelled in a different case passed both rails and wrote the redacted copy inside the live vault. The rails now test containment across several path spellings and refuse if ANY of them reports containment in EITHER direction, so the guard can only ever tighten. A symlinked vault — this project's own layout — is covered explicitly by test.
  • A new way of finding these defects. Most fixes in this release came from reading code and tests. This block came from two additional passes that don't depend on either: a wide static "honesty census," scanning the tool surface for the recurring shapes seen elsewhere in this release (an unverified object scope, a single page of results mistaken for a complete scan, an absence read as a checked-clean zero); and, later in the release, seven personas asking a real, 9,264-node production-shaped org real natural-language questions through the actual server, choosing tools themselves the way a host LLM would. An independent adversarial grader re-verified every finding from that second pass against the live vault before it counted — 27 of the 87 candidate findings did not survive re-verification and were dropped, 60 did, and 9 of those were rated critical. Both methods exist for the same reason: this product's own automated test suite runs against fixtures, and a fixture that was never built to contain a given real-world shape cannot report that the product mishandles it — which is exactly the shape of all four defects fixed in their own dedicated pass below.
  • The clearest illustration: three different tools gave three different delete-safety answers about the same 427 fields, in one session, against one vault. Asked which custom fields were unused, the org-wide tool answered 427, every one at high confidence and marked complete with no caveats. The product's own per-field delete-safety tool, run over those same 427 fields, returned a hard "blocking" verdict for 16 of them; the raw Flow metadata showed 33 of them are written by Flows that are currently active. The mechanism: a separate scan of Flow field-writers that the per-field tool already consulted and the org-wide tool did not, so "no incoming edges" was literally true of the edge table this tool reads and materially false about the org. The org-wide tool now runs the same scan before answering, plus a cross-check against report-type columns, and states plainly when either check could not run — rather than certifying a deploy-ready deletion list it had not actually earned.
  • A one-word property mismatch hid every Process Builder from the migration tool built specifically to find them. The tool filtered candidates on a boolean property that no Flow-family node in this product has ever actually carried — the extractor writes a different property name for status — so the default filter silently dropped all 275 real candidates and reported a certified, uncaveated zero. Forced open, the one real candidate it then surfaced was stamped inactive, directly contradicting its own source metadata. Status is now read from the property the extractor actually writes, as a three-state value where "never had a status property at all" survives the filter instead of silently collapsing into a zero.
  • A "checked zero" was worded to specifically remove the hedge a careful reader would otherwise add. One tool answered a metadata type with "0 unused, because this org holds none of this type at all — the 0 is a checked zero" — while the graph held 79 real edges from other components naming 30 distinct members of that type that were simply never retrieved. Separately, a classification bucket covering protected-class personal data (race, religion, disability, veteran status, and similar categories) was silently excluded from the audited population by three different downstream reports, with nothing in any of their limitations disclosures saying so.
  • The reasoning engine could report a rule as fully satisfied when its own evidence gathering had been cut short. A fan-out cap bounding six evidence walks recorded nothing when it actually triggered, so a rule whose supporting citations were truncated by the cap was reported as satisfied rather than unevaluable; the engine already knew how to attach a coverage caveat to a truncated result, it simply was never told truncation had happened. The same pass also found that a family of tools consulted before deleting things could not distinguish "this Apex was scanned and came back clean" from "this Apex was never scanned at all," because both states collapse into the same empty result — so a vault built before the relevant quality recognizer shipped, or any refresh that skipped Apex, reported full static coverage over a corpus nothing had actually examined. This is the same class of bug the product had already fixed once for Apex triggers on the field-security side, left standing here.
  • The rest of that critical list, briefly: a compliance report's own advice for recovering truncated results pointed at an argument its advertised schema didn't actually have, and its headline "zero exposure pairs" result came from auditing only the first 25 of the regulated fields in scope; a method-level test-coverage tool certified full, complete coverage for a method reachable only through a trigger, against zero actual coverage evidence on the graph. One of these fixes was later caught as incomplete by a second, independent review pass: the compliance report's exposure count moved from 0 to a real number, but the underlying audited population still excluded the same protected-class fields by construction — the real fix was adopting a shared classification check the product already had, written specifically to prevent this exact omission, which the report had built its own narrower version of instead.
  • Descriptions turned out to be a routing surface, not just documentation — measured, not assumed. Thirty-four tool descriptions were rewritten across these fixes to match handlers that had changed, and three tools measurably got harder for a host to find as a result of being made more honest: one tool's self-recall in the routing funnel fell by a third when its description swapped concrete, searched-for nouns for vaguer language; another's fell by half when it stopped naming the exact phrases people search with ("dead code," "flows deployed but never run"); a third dropped out of the top results for a real query when an honest new section more than doubled its description's length and diluted an unrelated match. The fix in each case was not to remove the added honesty — it was to say the same true thing in fewer, more findable words.
  • Three tests in the integration suite had been failing since the product's early 0.1.x releases, and nobody had seen it, because CI excludes that whole directory by design. All 18 tests in tests/integration/end-to-end.test.ts pass now, for three different reasons. One asserted the refresh manifest's version equalled the literal string '0.1.0' — true the day it was typed, false at every release since, now derived from the CLI's own version at read time. One was a hand-written list of 141 realistic tool inputs asserting exact coverage against the tool roster, meant to "fail loudly rather than silently miss coverage" as the product grew — the roster grew to 217 tools and the list did not, so 76 tools (including find_component_usages and who_can_access_object) were never exercised by this test at all, and the loud failure it was designed to produce landed in a directory CI never runs. The list is now generated from the full roster via schema-driven argument derivation, with the original 141 entries kept as overrides — real per-tool knowledge a blind derivation cannot supply — rather than treated as the whole census. The third was not rot: it asserted at least one Apex edge came from the older regex-based scanner, which a later, parser-grade AST pass (now the default) supersedes on any fixture simple enough to fully resolve — so the count legitimately went to zero on this test's tiny fixture. Verified against a real 9,264-node vault before rewriting, rather than assumed away: both extraction paths are alive there (1,844 AST edges vs 1,257 scanner edges), and the scanner alone still contributes real edges specifically out of Apex triggers. The test now asserts the actual invariant instead — every edge must cite the confidence of the mechanism that produced it — per source, over whichever paths actually fired.
  • The roster-wide sweep meant to catch exactly the defect family the last release apologized for could not have caught any of them — and fixing it immediately turned up 22 real violations. The sweep's core check recognized only two hardcoded stub markers and compared a tool's error response as a plain string against what live handlers actually return (a structured error object), so it could never fire on a real tool at all. Its coverage was also only 141 of 217 tools, hand-written the same way the dead 0.1.x sweep above was. It is now derived from the full advertised roster, with arguments generated from each tool's own schema, and it asserts three honesty rules directly on the final response a caller would actually see rather than on a handler's internal return value — checking the return value instead of the final response is exactly how a prior release's defect survived its own tests. The three rules: an empty result must carry a typed marker saying so; a paginated result's "more rows exist" flag and cursor must describe the rows actually shipped; and an unresolvable object scope must be refused, never silently answered. Each rule was proven to actually catch something before being trusted, including a case shipping 109 of 218 real rows while claiming nothing more existed. Running it for real found 22 tools failing (17 on scope refusal, three of which fabricated rows for objects that do not exist; 3 on typed absence; 2 on pagination) — recorded as known-red rather than quietly exempted. One was fixed immediately: a thin "field cleanup" alias over a deeper tool computed its own truncation flag from an already-paged slice and so always reported itself complete even when the underlying page said otherwise — one output format of an identical call said "complete," three others correctly said "more rows exist." Separately, the actual release gate script was found to name five test suites, two of which do not exist; the test runner silently treats an unmatched filter as nothing-to-run and exits successfully, so the release gate had been reporting green while running three of the five suites it claimed to run.
  • One of the 22 flagged tools turned out to be the check being wrong, not the tool. A tool that reports on Setup Audit Trail history was flagged for "answering about a nonexistent object," but its actual disposition — "this capability is switched off, here's how to turn it on" — is a more specific and more useful refusal than "that object isn't here," and forcing an object check first would have replaced a better answer with a worse one. It is now a narrow, counted exemption rather than a silent pass, bounded so it can never be mistaken for the rule itself passing. With that one carve-out in place, scope refusal now holds across all 58 object-scoped tools exercised by this sweep — up from 17 failing when this line of fixes began.
  • The guard against reintroducing single-page-scan bugs was failing on its own paper trail. Three tools that migrated away from a truncated corpus scan this release each explained the migration in a code comment naming the old call they had replaced — and the guard, which matches that old call's name anywhere in the file, failed all three simply for saying so in a comment. It now strips comments before matching.
  • Data loss: reconciling a small metadata family against an empty retrieve could silently delete every file the vault held for it. The mechanism that removes vault files for components no longer present in a fresh retrieve has two independent safeguards meant to prevent wiping an entire metadata family in one pass — and both could be stepped past at once. One safeguard only engaged when the retrieve had returned some files of the affected type; the other didn't engage at all below 20 files under consideration. So a retrieve that produced nothing of a given type, against a vault holding fewer than 20 components of it, deleted all of them and reported success — an identical 100% deletion was refused at 20 files considered and performed at 19, a cliff rather than a policy, reproduced before fixing. The code comment directly above the 20-file threshold asserted this exact case "is still caught by the [other] rail, which is size-independent" — it was not; the comment was standing in for a guard the code did not actually have. A total wipe of a family now refuses at every size, regardless of whether the retrieve returned any files of that type at all, since an empty retrieve result is equally the shape of a genuinely-emptied org, a failed retrieve, or one read in a layout the tool doesn't recognize.
  • The previous release's fix for a false-positive in find_component_usages introduced the opposite bug for Visualforce. The earlier fix excluded a component's own bundle directory from counting as "something else uses this" — correct for component types where each component lives in its own directory, but Visualforce pages are flat, with every page in the org sharing one directory. Excluding the whole directory therefore discarded every real Visualforce caller, making the tool wrongly report no evidence of use for a page that other pages genuinely embed. Reproduced before fixing, then split into two separate rules — one bounding the file search, one deciding what counts as "the same component" for self-match exclusion — so a flat-layout component now excludes only its own file, not its neighbors.
  • sfi.resolve — the first tool in the core lookup chain, and the one nearly every other answer routes through — could confirm a component the live vault does not actually contain. The persisted lookup index used to validate itself by node count alone. A refresh builds into a temporary file and swaps it over the live database; when that swap fails, a stale index from the losing build can be left behind, and if it happened to have the same node count as the real one, it was accepted as valid — so a lookup could return a confident, high-scoring match for a class that is not in the vault. Worse, and not the failure mode anticipated going in: the same stale index also made the live vault's own real components briefly unfindable, since it was a false-negative machine for everything the losing build didn't contain, not only a false-positive machine. The check is now content-addressed — a fingerprint over exactly the data the index actually depends on, computed inside the database itself in about 12 milliseconds on a real 9,264-node / 107,864-edge vault — and an index built before this fingerprint existed is now rejected outright rather than silently trusted.
  • A truncated evidence slice in the reasoning engine was enumerating access grants at full confidence. The coverage caveat meant to accompany a claim built from a partial (cut-short) graph scan was wired to the wrong condition — whether the underlying rule described an absence, not whether the scan itself had actually been truncated — so a claim listing which fields a profile can read kept its full confidence and no caveat even when the scan behind it had been cut short. Measured on a real org: two field-level access rules over the identical truncated scan, in the same run, both fully confident and uncaveated while citing 807 and 641 grants respectively, against a real total of 2,659 readable and 2,123 editable fields — 81 of 285 relevant objects were dropped from the enumeration wholesale, and the list stopped mid-alphabet inside another. An admin asking whether a late-alphabet field is readable would get a confidently enumerated list that omits it and concludes no. Confidence is now capped to reflect truncation regardless of the rule's shape, and the claim text itself now states the coverage floor plainly ("this is at least this set, not necessarily the complete one") rather than leaving the caveat to a separate field a host could silently drop.
  • A natural-language evaluation harness was grading its own bug as a product failure. Run against a real 9,264-node vault, this internal testing tool reported half of its offline test questions failing — including questions the product advertises on its own front page, such as "what happens when a Case is created?" None were real product failures: the product's own question router already returns the concrete arguments a question implies (an explicit event type, a resolved subject component), and the harness's own argument-building step was ignoring that output and constructing arguments from the tool's schema alone — so the correct tool was called without the arguments the router had just supplied, answered honestly that it couldn't proceed, and the harness recorded that as a product bug. Verified by hand first: called the way the router actually instructs, the same questions return full answers. The harness now honors the router's supplied arguments; its own measured pass rate on a 30-question set moved from 8 to 12 correct as the fix landed, none of it a change in what the product itself answers.
  • The official MCP Registry record shipped the exact install defect the editor plugin already carried. The server resolves its vault location in order — an explicit flag, then an environment variable, then a relative default — and that relative fallback is relative to wherever the host happens to launch the process, almost never the user's actual Salesforce project. The manifest the release workflow publishes to the official MCP Registry on every tag declared no vault-related argument at all, so the one discovery channel with any real evidence of delivering a non-maintainer visitor installed a server that would start with no org configured. It now declares the vault path as a required, user-supplied setting, so an installing host prompts for it; the editor-plugin manifest, which by nature cannot ship a user-specific absolute path, instead keeps a written, gated exemption and points to the built-in setup-status tool. The same commit fixed the security policy document, which had listed an old release line as the current supported version three releases after the version that followed it shipped — the one page a security researcher reads first — and both are now checked by the release-consistency gate rather than left to a human to remember.
  • Every install instruction on the npm package page — the page a stranger reads before installing — was broken, verified against the live registry rather than assumed. The primary registration command and the desktop-client config example both omitted the vault path, which meant the documented command produced a server pointed at a vault that does not exist. The "first run" section opened with the init command and no target-org flag, which a prior release had made required whenever the command isn't run interactively — true of every MCP host — so the very first command on the page failed in exactly the environment it was telling readers to set up. There was no section for one major editor host at all, despite that host keying its config differently from every other one documented (a difference the root project README had already covered, just not the page people actually install from). And the offline demo command — the only path needing no org, no authentication, and no existing Salesforce project, which is everyone arriving from npm — was absent entirely. The demo now leads the page, every registration example carries an absolute vault path, a host-comparison table makes the config-key difference impossible to miss, and two new sections cover "connected, but the org looks empty" and how to leave feedback. Every command on the page was checked against the source code that implements it rather than copied from older prose.
  • The screen a first-time user reaches when setup is broken had no way to say so. The tool reached whenever the server starts and finds no vault — the single moment a first run is most likely to be failing — offered a documentation link and nothing else. This project's GitHub issues have been open and unrestricted for ten weeks with zero received, which reads as "no users" but is at least partly "no channel for the users who never got far enough to file one." The tool's guidance now includes a way to say something went wrong, worded deliberately as an invitation ("the setup path getting something wrong is the likeliest cause") rather than a bug-report demand, since at this stage the probable fault is the product's own instructions, not the reader's machine.
  • The machine-readable self-description handed to every AI crawler and answer engine that asks was a release behind. It still named the previous release two days after the current one had shipped as the published npm package, and its companion long-form file said the same; the script that fixes this already existed and simply hadn't been run at release time. The version string is now checked by the release-consistency gate, so a release can no longer ship disagreeing with its own website. Run for the first time against a fully built tree at this release, that same script also corrected the published unit-test-case and integration-suite counts — and, worth recording plainly, a prior release note by this project had accused the website of understating its own test count against a much larger figure in a commit message; running the authoritative counter shows the website was approximately right and the commit-message figure was the outlier, since a full test run counts every parameterized expansion where the site counts test cases from source. Not the staleness it was originally reported as.
  • A green publish job could leave the official registry pointed at an old version while npm moved on. The registry-publish step in the release workflow tolerated its own failures silently, so any registry-side problem was swallowed and the workflow still reported success — meaning the record every MCP directory and host reads to discover this server could quietly fall behind the npm package. That tolerance wasn't accidental: without it, re-running a partially-failed publish would die on the registry's hard rejection of a duplicate version, after npm already had it, stranding the release before it could reach the remaining steps. The fix tells the two failure cases apart instead of ignoring both: an "already published" response is now treated as an idempotent, successful re-run, and every other failure is fatal and says plainly that npm and the registry are now out of sync.
  • Thirteen tools answered privacy, sharing, exposure and delete-safety questions about metadata objects the vault had never found — with no marker distinguishing a real answer from an unchecked guess. A derived sweep called every one of the product's 217 tools with a nonexistent object name and found the same shape repeatedly: a PII inventory tool answered "0 personal-data fields" for an object that was never in the vault; a list-view sharing tool answered "nothing is shared"; a sharing-summary generator rendered a full document; a domain-clustering tool answered clean — all with no boundary, no disclosure, nothing distinguishing "we checked and found none" from "we never found the object." For a privacy or sharing tool, a clean zero about something that does not exist reads as "no personal data on this object" or "nobody is sharing it," not as "this tool never actually checked." The same unverified string-concatenation into a graph id also meant a real object typed in the wrong case answered exactly as emptily — a lowercase spelling of a real, common standard object returned "no personal data" from a tool that should have found real matches, and "nobody is sharing it" from a tool that should have found real sharing rows: a defect a user hits by typing normally, not an edge case. Five more tools — covering record-creation paths, Change Data Capture subscribers, event topology, setup-audit attribution, and field-history tracking gaps — built object ids from raw string transforms with the identical gap (one of them case-sensitive on top of it). Three more — the guest-user exposure report, the AI-surface exposure report, and the value-change audit — answered "the internet cannot see this" or a clean all-zero summary for an object that was never modeled, which for an exposure tool is the one wrong answer that matters most. All now resolve the object against the graph before answering, through the module the product already used for this in five other places, rather than growing a sixth ad-hoc spelling of the same check. Several needed real judgment rather than a mechanical swap: two tools deliberately still answer for a standard object whose automation was retrieved even though its own object metadata was not (a real, distinct case a naive existence check would have started wrongly refusing); the setup-audit tool validates after an unrelated capability check that is honest regardless of the object's existence; and the record-creation-paths tool takes a required object and has no org-wide mode to begin with.
  • find_dead_code named a real, existing Apex class deletable under an object the vault does not contain. Its object-scope filter applied only to custom-field rows — Apex classes, triggers and Flows were scanned org-wide and then shipped labeled as scoped to the (nonexistent) object. Measured on the demo vault: scoping to an object that isn't real returned a genuine class marked "likely dead" with a full type breakdown, attributed to that nonexistent object — and the same leak fired on every real scoped call too, silently mixing in candidates from the whole vault rather than the object actually asked about. Two related advisory tools had different but similarly shaped defects on the same class of question: one fabricated a "this object is greenfield, here's a recommendation to build automation on it" response for an object that isn't real; the other ignored the scope argument entirely and returned the full org-wide briefing, byte-identical to an unscoped call. All now resolve the object first and refuse what isn't there — with one deliberate exception, since a standard object like Case has real automation with no dedicated object-metadata node of its own, so that check also accepts an object visible only through edge evidence.
  • Security: the component-documentation fallback used by sfi.get_component could be made to read Markdown files outside the vault. The tool passes a caller-supplied id straight into its documentation-fallback lookup, which parsed it by splitting on the first . and joined the pieces into a filesystem path with no rejection of anything unusual in them — so a crafted id could climb outside the vault entirely. Confirmed by execution rather than by reading the code alone: with the containment check disabled, a regression test plants a Markdown file outside the vault and the unpatched tool reads it back successfully. The disclosure half was worse than the read itself: the relative-path computation on the way out fell back to the file's full absolute path whenever the resolved location wasn't inside the vault, so on exactly the inputs that succeeded in escaping, the response would have handed back the server's real filesystem layout even when the read itself failed. Containment is now checked structurally — the candidate path is resolved and then tested for whether it is still lexically inside the vault root, rather than trying to deny-list every possible spelling of a traversal sequence — and the check deliberately does no filesystem I/O and does not resolve symlinks, so a symlink planted inside the vault that points outside it still cannot become a read. This is reachable from ordinary tool arguments with no local access to the machine, and it is pre-existing, not a defect introduced this release — present since the 0.3.0 line. The practical exposure is narrower than the missing validation itself, since the read only succeeds for paths ending .md that survive a further parsing step — but the missing validation was total.
  • The end-to-end MCP smoke test hardcoded the exact roster count this same release changed, and nothing local ever ran it to notice. packages/mcp/e2e-smoke.mjs asserted the core profile advertises exactly 19 schemas and used sfi.org_overview as its example of a tool that is NOT directly invokable under core — both literally true until this release's roster-expansion fix (core grew to 25 tools, org_overview among the six newly promoted). The script is not part of the vitest suite and nothing in this project's local gate list runs it, so this drifted silently through every local test run for the rest of the release and was only caught on the first real push to main, where CI runs it for real. Both assertions now derive from the same CORE_PROFILE_TOOLS set the server itself gates on — the expected count, and the non-core example tool used to prove denial — imported from the package's own public API rather than restated as a second, driftable copy.

Verification scope

Windows: partially verified. The hard-gated tier (core, tooling-api, renderers, parsers, patterns, vault, extractors) passed. Advisory tiers were NOT green: WIN-1=failure, WIN-2=failure — those suites did not report a pass on Windows for this build and were not allowed to block it. See CI run https://github.com/PranavNagrecha/Salesforce-Intelligence/actions/runs/37152242522.

CI run: https://github.com/PranavNagrecha/Salesforce-Intelligence/actions/runs/37152242522