Skip to content

fix(safety): add pyproject.toml parser support for uv/poetry Dependabot PRs#67

Merged
j7an merged 24 commits into
mainfrom
fix/pyproject-toml-parser
May 24, 2026
Merged

fix(safety): add pyproject.toml parser support for uv/poetry Dependabot PRs#67
j7an merged 24 commits into
mainfrom
fix/pyproject-toml-parser

Conversation

@j7an
Copy link
Copy Markdown
Owner

@j7an j7an commented May 24, 2026

Summary

  • Adds scripts/pyproject-bump-extract.sh, a diff-aware helper that extracts dep rows from pyproject.toml bump-only edits and clears those paths from the fail-loud guard.
  • Updates dependency-cooldown.yml and dependency-safety.yml to compose the helper into the existing pipeline and introduces EFFECTIVE_TOUCHED so the Layer 3 zero-row guard does not fire on comment-only pyproject churn.
  • Path-only classify-touched-paths.sh is unchanged in behavior; conservative path classification is preserved.

Test plan

  • bats tests/ — full suite green (201 tests), including the new pyproject-bump-extract.bats (43 cases) and the extended dep-guard-chain.bats (8 new integration tests covering AC1–AC4 + cross-helper dedup + zero-row regression).
  • ./scripts/check-inline-sync.sh — inline copies match scripts/pyproject-bump-extract.sh byte-for-byte in both workflows (14/14 OK).
  • ./scripts/lint-workflow-call.sh — no caller-context refs.
  • ci-safety.yml — workflow runs against this PR's own Dependabot-style diff fixtures (smoke-test the helper composition on real CI).

Fixes #66.

j7an added 24 commits May 23, 2026 20:06
Adds the helper skeleton with flag parsing, empty/malformed input
handling, and the bats test contract. Recognition logic for actual
pyproject.toml hunks lands in subsequent commits.

Refs #66.
…bumps

Implements per-file state machine, table/key tracking, PEP 508
string-in-array bump-pair recognition, and target-version extraction
(target-bearing operators only). Other recognized tables follow in
subsequent commits.

Refs #66.
…nt/override

All four use the PEP 508 string-in-array form; the existing state
machine routes them via table-header recognition. Adds fixtures and
assertions only.

Refs #66.
Adds fixtures and tests for the canonical Poetry bump shape and the
python-constraint disqualifier. The Task 2 parser already supports
both via the centralized pending tracker and the python-key check
in the poetry_main|poetry_group|poetry_dev branch.

Refs #66.
Inline-table form (`name = { version = "spec", ... }`) is handled by
the Task 2 parser via skeleton comparison: the inline body has
`version = "..."` substituted with a sentinel and both sides must
match byte-for-byte. Locks behavior with version-only-bump positive
and extras-change negative.

Refs #66.
Both route through the existing poetry key=value/inline-table logic
via the state-machine's table-header recognition.

Refs #66.
Subdir paths surface as full b/... paths in cleared-paths output;
verdict accumulation is per-file (one disqualified file does not
affect a cleared sibling).

Refs #66.
Covers every disqualification path:
- add/remove/marker/extras-only changes
- version+marker and version+extras combined changes (skeleton mismatch)
- unmatched - lines followed by context (pending-lifetime enforcement)
- structural changes: adding an extras key, adding the dependencies = [
  array, adding/removing table headers
- build-system, unrecognized tables, mid-array without header context

Every test uses assert_disqualified which checks BOTH --mode=deps
and --mode=cleared-paths emit empty output — checking only one
cannot distinguish a disqualified file from a clean comment-only
file.

Refs #66.
Per §3.4 rule 2, comment-only and whitespace-only changed lines are
harmless and may appear anywhere in the diff. The path clears with
zero extracted deps; downstream composition prevents this from
tripping the Layer 3 zero-row guard (see EFFECTIVE_TOUCHED in Task 12).

Refs #66.
…action edge cases

Spec §3.3.1 specifies that bumps with unchanged PEP 508 markers
containing backslash-escaped double-quotes (e.g.,
"foo>=1.2; python_version < \"3.12\"") must extract normally.
The parse_pep508_entry regex's marker portion was [^\"]*, which
rejected this form and disqualified it via the catch-all path.

Loosen the marker portion to .* — bash regex greediness + the trailing
anchor pattern still force the correct closing quote. The 13 Task 8
disqualifier fixtures continue to disqualify (some via skeleton
mismatch instead of unrecognized-line, same observable outcome since
assert_disqualified checks both modes are empty).

Positive: digit-bearing names, unchanged markers (incl. escaped),
PEP 440 post-releases. Negative: compound specs, upper bounds,
not-equal, wildcards. Operator allowlist is target-bearing only
(^, ~, ~=, >=, ==, none).

Refs #66.
…ader

No behavior change. Notes that the workflow composition layer may
clear pyproject.toml paths via the new diff-aware helper, while the
classifier itself remains path-only.

Refs #66.
…ring update

Mirrors the 6-line composition-layer note added to
scripts/classify-touched-paths.sh in the previous commit so
check-inline-sync.sh passes.

Refs #66.
…y-safety

Embeds pyproject_bump_extract as an inline function, computes
CLEARED_PYPROJECT in the composition hoist, and updates Layer 3's
zero-row condition to use EFFECTIVE_TOUCHED so cleared pyproject
paths do not trip the guard. Layer 2 comment updated to reflect the
new clearance pattern. guard-runtime.bats updated to supply
EFFECTIVE_TOUCHED alongside TOUCHED_PATHS for safety workflow tests.

Refs #66.
Same four edits as the previous commit, applied to the legacy
cooldown workflow that's retained for the Phase 2 migration window.
Embedded function body is byte-identical to the safety workflow's.

Refs #66.
Adds the two new INLINE_PAIRS entries so check-inline-sync.sh fails
loudly if scripts/pyproject-bump-extract.sh drifts from the inline
copies in either workflow.

Refs #66.
Adds run_chain_with_pyproject helper that mirrors the workflow's
full composition (including EFFECTIVE_TOUCHED), updates the existing
pyproject-only assertion to reflect new clearance behavior, and adds
acceptance-criterion coverage for AC1–AC4 and cross-helper dedup.

Refs #66.
…th-only)

Adds a comment noting that the workflow composition may clear this
path via the diff-aware helper. Prevents a future reader from
"fixing" this isolation test to reflect composition-level behavior.

Refs #66.
…DE.md

Updates the workflow-roles description so the inline-embed listing
includes the new helper alongside the existing four/five scripts.

Refs #66.
The four `cooldown:` test cases in `tests/guard-runtime.bats` were not
updated when the cooldown workflow's Layer 3 elif switched from
`$TOUCHED_PATHS` to `$EFFECTIVE_TOUCHED`. The `safety:` counterparts
got the new variable; the cooldown ones did not, so the issue #52
cooldown case began evaluating an unset `EFFECTIVE_TOUCHED` and the
guard branch never fired. CI caught the regression even though local
test output buried the failure.

Mirrors the same pattern the safety tests use: bind
`EFFECTIVE_TOUCHED` to `$TOUCHED_PATHS` (or empty when both should be
empty), reflecting the workflow's composition behavior when no
pyproject paths were cleared.

Refs #66.
PR review found five paths where the parser would incorrectly extract
or clear a pyproject.toml edit that should remain unsupported:

1. current_key leaked from [project].dependencies into a subsequent
   keywords = [ array (and equivalent in [tool.uv] for any non-
   constraint/override key), causing later PEP 508 entries to be
   treated as in a dependency array and silently cleared.
2. parse_poetry_inline matched 'version' anywhere in the inline body,
   including inside 'subversion', misclassifying unsupported inline
   tables as version bumps.
3. emit_bump compared only literal version specs and target versions
   — operator changes (==1.0 → >=2.0) passed as bumps even though
   they are semantic constraint broadenings.
4. The inline-table skeleton substitution replaced the entire
   'version=...="..."' clause with a fixed form, normalizing away
   whitespace differences around '=' and masking reformatting+version
   changes.
5. Bare PEP 508 strings (e.g., 'foo 1.0') with a space instead of an
   operator were accepted, even though PEP 508 requires an operator
   before the version.

Fixes:
- Reset current_key when a [project] or [tool.uv] key is not a
  recognized dep-array key (covers context lines too).
- Anchor 'version' in parse_poetry_inline to '^|,|whitespace' both
  in the match regex and the skeleton sed substitution.
- Add extract_operator helper and require operator byte-equality
  before extracting versions in emit_bump.
- Preserve original surrounding whitespace in the inline skeleton
  substitution so reformatting+bump is detected.
- Validate that PEP 508 version_spec is empty or starts with a
  recognized operator in parse_pep508_entry.

Adds 8 negative regression fixtures + tests using assert_disqualified
so both --mode=deps and --mode=cleared-paths are checked.

Refs #66.
…r fixes

Mirrors the parser changes from the previous commit into the inline
copies in both workflows so check-inline-sync.sh stays green.

Refs #66.
PR review (Bug 1b) found that the previous fix only reset current_key
when a non-dep-array key line was visible. If a recognized dependency
array closed and was then followed by PEP 508-shaped lines without a
fresh array-opening key in context (malformed/truncated hunk), the
parser still treated those strings as dependency entries and could
clear the path silently.

Reset current_key on any closing `]` line — both +/- (which already
disqualify as reformatting) and context. After a closing `]`, any
later in-array detection requires fresh context to re-establish.

Adds pep621-current-key-leak-post-close.diff with the exact
reviewer reproducer and an assert_disqualified test that fails
without the fix and passes with it. Inline copies in both workflows
re-synced.

Refs #66.
Reduces fixture-file noise in PR #67 by moving 31 small unit-test
diffs from tests/fixtures/pyproject-bump-extract/*.diff into literal
heredocs inside tests/pyproject-bump-extract.bats. Keeps canonical
positive fixtures and integration fixtures external (where literal
files improve reviewability or are reused).

Adds run_pyproject_deps, run_pyproject_cleared,
assert_disqualified_diff, and assert_clean_bump_diff helpers that
accept a diff string instead of a file path. Existing file-based
helpers (assert_disqualified, assert_clean_bump) preserved for the
14 external fixtures still in use.

Test names and assertions are unchanged; verified bats tests/ still
reports the same count and zero failures.

Refs #66.
Three new helpers in tests/pyproject-bump-extract.bats build minimal
diffs from minus/plus arg pairs:
- pep621_deps_diff       — PEP 621 [project] dependencies single ±
- poetry_main_kv_diff    — Poetry tool.poetry.dependencies keyval
- poetry_inline_diff     — Poetry inline-table single ±

15 disqualifier + 3 positive tests that previously used ~10-line
heredocs now read as one-line builder calls with the minus/plus
content as args. Test names and assertions unchanged. Tests with
asymmetric or multi-line hunks keep their heredocs since the
builders do not fit.

Refs #66.
@j7an j7an merged commit c48eba3 into main May 24, 2026
7 checks passed
@j7an j7an deleted the fix/pyproject-toml-parser branch May 24, 2026 07:16
j7an added a commit to j7an/cross-agent-reviews that referenced this pull request May 24, 2026
Picks up upstream fix for pyproject.toml parser support
(j7an/shared-workflows#67), unblocking uv/poetry Dependabot PRs at the
dependency-safety / gate.

v3.0.0 migration checklist (j7an/shared-workflows#69 removed legacy
cooldown.yml paths) — already satisfied here:
- Native cooldown.default-days in .github/dependabot.yml: yes (5)
- Caller uses dependency-safety.yml: yes
- Input minimum_release_age_days: yes
- No fail_on_cooldown / cooldown-rescan usage: confirmed
- Required status check name dependency-safety / gate: unchanged

Refs #38.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

extract-deps.sh: add pyproject.toml parser support (uv/poetry Dependabot PRs currently error at the gate)

1 participant