Skip to content

fix(signals): subject over-long manifest entries to dedup and list cap#524

Merged
JSONbored merged 3 commits into
JSONbored:mainfrom
philluiz2323:fix/focus-manifest-overlong-dedup-cap
Jun 11, 2026
Merged

fix(signals): subject over-long manifest entries to dedup and list cap#524
JSONbored merged 3 commits into
JSONbored:mainfrom
philluiz2323:fix/focus-manifest-overlong-dedup-cap

Conversation

@philluiz2323

Copy link
Copy Markdown
Contributor

Closes #522.

normalizeStringList truncated an over-long entry (> MAX_ITEM_LENGTH 300) then continued, skipping both the !result.includes de-duplication and the MAX_LIST_ITEMS (200) cap that the normal path enforces. Manifests are fetched from untrusted, arbitrary repos, so a crafted .gittensory.yml/.json defeated both limits:

  • duplicate entries: two over-long entries sharing a 300-char prefix both pushed after truncating to the same string;
  • cap bypass (reachable): a list of all over-long entries never hits the break; 201 × 301-char entries (~61.5 KB) fit under the 64 KB MAX_FOCUS_MANIFEST_BYTES gate, yielding a 201+ entry list past the 200 cap.

Change

  • Truncate over-long entries in place, then fall through to the same de-dup and cap logic (no continue).
  • Add tests: over-long entries sharing a 300-char prefix de-duplicate to one entry; 250 over-long entries cap at 200 with the exceeded 200 entries warning. Both fail on the old code.

Verification

  • focus-manifest.test.ts 65/65; tsc --noEmit clean; changed file branch coverage 98.36% (≥ 97% gate). (The one local-only failure in gittensory-focus-manifest.test.ts is a pre-existing Windows CRLF artifact comparing the on-disk .gittensory.yml; it passes in CI on LF and is unrelated to this change.)

Same family as merged #464 (dedup-after-cap), different site/defect.

normalizeStringList truncated an over-long entry then continue'd, bypassing both the !result.includes de-dup and the MAX_LIST_ITEMS cap -- so untrusted manifests could produce duplicated and over-cap (201+ entries fit under the 64KB gate) wantedPaths/blockedPaths/etc. Truncate in place and fall through to the same de-dup and cap logic.
@ghost

ghost commented Jun 10, 2026

Copy link
Copy Markdown

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal Result Evidence Action
Gate result ⚠️ Skipped #524 is no longer open. No action.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 10, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jun 10, 2026

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. This is a correct, well-scoped fix for a real input-validation bug.

On main, normalizeStringList (src/signals/focus-manifest.ts) truncates an over-long entry, pushes it unconditionally, then continues — so an over-long entry skips both the !result.includes de-dup and the MAX_LIST_ITEMS (200) cap break. Since manifests are parsed from untrusted arbitrary repos under the 64 KB MAX_FOCUS_MANIFEST_BYTES gate, a crafted .gittensory.yml/.json of all over-long entries bypasses both limits (e.g. 201 x 301-char entries fit under 64 KB and yield a 201-entry list past the 200 cap; two over-long entries sharing a 300-char prefix both land after truncation).

The fix truncates in place and falls through to the shared de-dup + cap path — minimal and correct. Tests cover both the de-dup-after-truncation case and the 250-entry -> 200 cap with the warning, and both fail against the old code. Same defect family as the merged #464, different site. CI green.

One optional follow-up (not blocking): the truncated value is the raw 300-char prefix slice, so two distinct entries that differ only after char 300 collapse to one — acceptable here since these are advisory path lists, but worth a one-line note if exact-path fidelity ever matters.

@dosubot dosubot Bot added the lgtm label Jun 11, 2026
@ghost

ghost commented Jun 11, 2026

Copy link
Copy Markdown

reviewbot · verdict: comment

Reviewed 2 changed file(s) — two independent AI reviewers:

Reviewer A · llama-3.3-70b-instruct-fp8-fast
This PR fixes an issue where over-long manifest entries could bypass de-duplication and list caps, and adds tests to verify the fix. The changes are clean and well-structured.

Worth double-checking

  • Double-check that the truncation logic does not introduce any edge cases with non-ASCII characters or Unicode normalization

Reviewer B · mistral-small-3.1-24b-instruct
This PR addresses the issue of over-long manifest entries by truncating them and subjecting them to deduplication and list capping. The implementation is clean and well-documented, with corresponding tests added.

Suggestions

  • Consider adding a comment explaining why MAX_ITEM_LENGTH and MAX_LIST_ITEMS are chosen as they are, if not already documented elsewhere.
  • Ensure that the truncation does not inadvertently remove meaningful parts of the entries, which might affect the functionality.

Worth double-checking

  • Verify that the truncation logic does not introduce any edge cases, especially with entries that are exactly at the MAX_ITEM_LENGTH.
  • Check if there are any performance implications with the new truncation and deduplication logic, especially with very large manifests.

@dosubot dosubot Bot added the size:S label Jun 11, 2026
@JSONbored
JSONbored merged commit 0a2fa54 into JSONbored:main Jun 11, 2026
7 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jun 11, 2026
@philluiz2323
philluiz2323 requested a review from JSONbored June 11, 2026 13:21
@github-actions github-actions Bot mentioned this pull request Jun 11, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

[Bug]: Over-long focus-manifest list entries bypass both de-duplication and the MAX_LIST_ITEMS safety cap

2 participants