Skip to content

review(public-safety): ordinary English ('reviewability', 'ranking', 'reward') still drops narrative sentences on every repo #9555

Description

@JSONbored

Parent: #9432

Summary

#9445 fixed the catastrophic half of #9432 — the narrative assessment is now filtered per sentence (toPublicSafeBySentence) instead of all-or-nothing, so a single flagged word no longer discards the entire summary.

A residual remains. FORBIDDEN_PUBLIC_COMMENT_WORDS is matched with a plain case-insensitive .includes(), and it contains ordinary English:

reward, rewards, ranking, rankings, cohort, farming, reviewability

Only bare score has a per-repo escape hatch (allowBareScoreTerm, driven by LOOPOVER_PUBLIC_SCORE_TERMS_ALLOWED_REPOS). Its siblings have none — so on every repo, allowlisted or not, a sentence like:

"This change improves reviewability and updates the ranking comparator so ties resolve deterministically."

is still dropped from the published narrative. That is a perfectly safe review of this codebase's own gate/scoring code, and reviewability in particular is unavoidable code-review vocabulary.

The result is no longer a missing summary (that was #9445) but a quietly incomplete one — arguably harder to notice, because the output looks fine.

Requirements

  1. Extend the existing per-repo exemption to the ambiguous ordinary-English terms, using the same allowlist and the same positively-confirmed-per-repo requirement. No new config surface, no blanket default.
  2. Terms that name a private concept outright must stay enforced for every repo, allowlisted or not.
  3. Default behaviour for a non-allowlisted repo must be byte-identical to today.

Why the split is safe (the load-bearing argument)

The risk this filter exists to stop is a leaked private value. A value never appears as a bare noun — it appears qualified:

  • reward estimate 12 TAO → contains reward estimate
  • raw trust score 0.82 → contains raw trust score
  • score preview, estimated score, private ranking, wallet, hotkey, coldkey, seed phrase, mnemonic, payout

Every one of those qualified forms belongs in the always-forbidden tier. A bare reward or ranking with no number attached leaks nothing on its own. This generalizes the judgment allowBareScoreTerm already made for score to its siblings rather than inventing a new one.

Deliverables

  • Split the vocabulary into ALWAYS_FORBIDDEN_PUBLIC_COMMENT_WORDS (never exemptible) and AMBIGUOUS_PUBLIC_COMMENT_WORDS (exemptible via the existing allowlist).
  • Keep FORBIDDEN_PUBLIC_COMMENT_WORDS exported and derived from both, so existing consumers and the redaction-parity test keep one canonical vocabulary.
  • Gate the ambiguous tier on the existing allowBareScoreTerm flag.

Tests

  • SECURITY, exhaustive over the real list: every always-forbidden phrase still throws even with the exemption on — asserted over the array itself, so a term moved into the wrong tier fails immediately rather than passing a hand-picked sample.
  • SECURITY: a qualified private value stays blocked for an allowlisted repo even though its bare noun is exempt (reward exempt, reward estimate 12 TAO blocked).
  • An allowlisted repo keeps a sentence using each ambiguous term.
  • Without the exemption, the ambiguous tier is still enforced — the default is unchanged.
  • The two tiers are disjoint and together exactly equal the combined list — no term silently lost or double-listed.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions