Skip to content

v0.1.24 — assignment-data engine + honesty R4

Choose a tag to compare

@PranavNagrecha PranavNagrecha released this 02 Jul 19:05
· 282 commits to main since this release

Headline: assignment-data engine + router honesty R4 + experimental embeddings.
Three simultaneous workstreams: four new live tools close the "who holds X / who's in
Y" honest-gap family; the router's fourth honesty round adds write-evasion hardening,
forecast/authorship gaps, narrowed clarification, and show-me candidate coverage; and
an opt-in RRF hybrid embeddings layer sits behind a feature gate for early adopters.
Tool count 172 → 176.

Measured on the maintainer's two real-org suites (2,000 + 2,995 primary
questions, 12,352 turns, 0 route errors), 0.1.23 → 0.1.24:
honesty rose on
both — declined-correctly 82.9 → 83.8% (2K) and 57.2 → 66.9% (3K, +9.7)
while over-routing fell on both (2K 89 → 88; 3K 229 → 185, −44), reversing
the eagerness regression 0.1.23 introduced. The injection/write-evasion family
dropped from 8 leaks to 1 (a read-only report with execution skipped, unchanged
since 0.1.23); the curated write-execution set is 24/24 refused. Answer-recall
held (2K 82.9%, 3K 80.2%), recall@3 61.1, needs-live 76.9 → 78.1%. The recall
gain came from precision this round, not more eagerness — the opposite of the
0.1.23 trade.

Added

  • sfi.live_permset_holders — who HOLDS a permission set, permission set
    group, or profile (kind: permissionSet | permissionSetGroup | profile | auto), answered from the live org. PSG-trap-aware: direct holders and
    via-group holders (PermissionSetGroupComponent) are reported separately
    with a deduped effectiveTotal, so the count is audit-grade instead of
    confidently understated. True count first, expired assignments excluded and
    disclosed, 500-row cap with byte-fit that never understates totals, keyset
    paging (afterId/nextAfterId), optional per-profile buckets. This also
    answers the name-by-name profile roster family.
  • sfi.live_user_permsets — the REVERSE direction: what a named USER
    holds. Direct permission sets vs via-PSG assignments (with expirations),
    profile named; PermissionSet.IsOwnedByProfile = false is pinned into every
    assignment query so the system profile-owned row never masquerades as a
    direct assignment. Pairs with vault sfi.effective_permissions for a
    dual-provenance answer (live = which grantors; vault = what they grant).
  • sfi.live_group_members — who is IN a queue / public group right now:
    users, nested groups (expanded at most ONE level, fail-closed and stamped
    expansion: 'partial-one-level'), role-based members surfaced as ROLE
    entries (never silently expanded), queue supportedObjects ("can this queue
    own Case"), and a measured vaultDeclaredMemberCount vs
    liveDirectMemberCount drift check.
  • sfi.live_zombie_accounts — active users with login access but ZERO
    permission-set/PSG assignments (single anti-join on
    PermissionSet.IsOwnedByProfile = false; disclosed bounded client-diff
    fallback when an org rejects the anti-join). Output states verbatim that a
    "zombie" still holds everything its PROFILE grants. Optional
    minDaysInactive / includeAllUserTypes. Dormancy-only questions stay on
    sfi.live_inactive_users.
  • All four follow the live-plane contract: consent-gated (sfi.live_consent /
    SFI_LIVE_PLANE_ENABLED / liveEnabled), budgeted
    (SFI_LIVE_QUERY_BUDGET, budget exhaustion is an honest error, never a
    silent fallback), read-only SOQL, provenance: live_org point-in-time
    stamps. No user identifiers land in the vault — the counts-only facts pin
    is untouched.
  • sfi.coverage_report assignmentData section — runtime assignment
    data (User / PermissionSetAssignment / GroupMember) is reported as
    "not in vault by design" (a runtime data object, not a retrieve gap),
    naming the four live tools, current live-consent state, and the counts-only
    facts snapshot presence/timestamp. sfi.health_check carries the same
    block informationally — it never degrades status; a >30-day-old counts
    snapshot earns an advisory only.

Changed

  • Router retargets (same change as the tools — no contradictory gates):
    permset-user-roster ("which users have permission set X") flips from
    honest-gap refusal to sfi.live_permset_holders; profile-user-roster
    drops its partial-answer gap (the name-by-name roster is now built);
    unassigned-permset-groups and permset-group-grants flip partially to
    sfi.live_permset_holders (per-PSG zero-holder check and PSG containment —
    the enumerate-all-PSGs sweep and the 2-hop "which PSG grants custom
    permission X" chain remain disclosed gaps); NEW queue-group-member-roster
    arm ("who's in the Support queue") routes sfi.live_group_members; NEW
    user-permset-holdings arm ("what permission sets does Jane have") routes
    sfi.live_user_permsets + sfi.effective_permissions as an ordered
    dual-provenance pair; empty-queues-groups keeps the vault scan primary and
    appends sfi.live_group_members for runtime verification.
  • The vault-side assignment disclosures (object_access_audit,
    who_can_access_object, and friends) now name the concrete live tools
    ("answerable via the live plane: …") instead of a generic "run the live org
    plane" pointer.
  • Router R4 — honesty + candidate coverage:
    Injection/write-evasion hardening (indirect re-delegation attempts and
    tool-self-capability asks refused with a read-side alternative);
    forecast/authorship honest-gaps (predictive "how will X change by…" and
    authorship/attribution asks return honest gaps naming the nearest real reads);
    narrow clarification re-introduced for genuine same-name collisions that
    the R2b rebalance over-suppressed; show-me candidate coverage (visual/UI
    render requests clarify to the relevant read tool rather than silently
    mis-routing).

Experimental

  • Embeddings hybrid (SFI_EMBEDDINGS=1, off by default) — an opt-in
    RRF hybrid layer that fuses the existing lexical TF-IDF candidates with a
    locally cached neural sentence-embedding model (Xenova/all-MiniLM-L6-v2,
    ~23 MB, downloaded once on first use from HuggingFace Hub into
    .sfi-embed-cache/). Affects candidate ranking only — the honesty/refusal
    decision and the route.tools deterministic plan are untouched.
    Graceful lexical fallback when the model is absent or the embed fails;
    allowRemoteModels is disabled, so the funnel can never phone home at
    query time. Requires npm i @huggingface/transformers in your project (not
    bundled). See docs/configuration.md §Embeddings for full opt-in details.