Skip to content

Match the HTTP link probe's open-PR guard on the real title prefix - #16421

Merged
Azat Mukhametshin (azat-msft) merged 3 commits into
microsoft:mainfrom
azat-msft:azat-msft-http-link-probe-guard
Aug 31, 2026
Merged

Match the HTTP link probe's open-PR guard on the real title prefix#16421
Azat Mukhametshin (azat-msft) merged 3 commits into
microsoft:mainfrom
azat-msft:azat-msft-http-link-probe-guard

Conversation

@azat-msft

@azat-msft Azat Mukhametshin (azat-msft) commented Aug 27, 2026

Copy link
Copy Markdown
Member

Problem

The probe's duplicate-run guard trusts a GitHub search count:

OPEN_PR_COUNT=$(gh pr list --state open --search '"[link-checker]" in:title' \
  --limit 1 --json number --jq 'length')
if [ "$OPEN_PR_COUNT" -gt 0 ]; then exit 0; fi   # skip dispatch

The intent is "does an open PR carry the [link-checker] prefix?". GitHub discards the brackets and matches the remaining words as a phrase anywhere in the title, so it really asks "does any open PR mention the words link checker?".

Measured against this repo on 2026-08-27 — these return an identical set, which is what shows the brackets contribute nothing, while reversing the word order returns none, which is what shows it is a phrase match rather than loose keywords:

Query Hits
"[link-checker]" 13
"link-checker" 13
"link checker" 13
"checker link" 0

Only 6 of those 13 carried the prefix. The rest are ordinary PRs that merely mention the checker, e.g. #16104 "Split the link checker workflow into HTTP and Relative link checker" and #16093 "Add agentic workflow link-checker".

The failure is silent and points the expensive way: the probe decides a checker PR is already open, skips dispatch, and reports success. The scan still runs, still finds changed links, and simply declines to act on them, for as long as the unrelated PR stays open. A guard meant to prevent duplicate work instead prevents all work.

This was live, not hypothetical. Open PR #16417 made the guard return 1, so the HTTP checker was suppressed.

Fix

The search is a superset of what we want, so keep it as cheap server-side narrowing and re-check the real prefix with jq:

OPEN_PR_COUNT=$(gh pr list --state open --search '"[link-checker]" in:title' \
  --limit 500 --json title \
  --jq '[.[] | select(.title | startswith("[link-checker] "))] | length')

Review surfaced two further holes in the same guard, both now closed:

Truncation. Filtering client-side means the rows must actually be in the response, and a short window is a wrong answer rather than a shorter list. gh pages through the API rather than capping at one page, so a low --limit is a ceiling we impose on ourselves:

gh pr list --repo NixOS/nixpkgs --state open --search '"update" in:title' --limit 250  ->  215
gh pr list --repo NixOS/nixpkgs --state open --search '"update" in:title' --limit 100  ->  100

That second line is silent truncation: 215 matches available, exactly 100 returned, no error. A prefixed PR pushed out of the window reads as "none open" and dispatches a duplicate. Limits are now 500 for the PR query and 200 for the run query; both stop early at the real result count (21 open PRs, 11 runs here today).

In-flight runs. The open-PR query cannot see an agent that has started but not yet opened its PR, so a probe run in that window dispatched a duplicate. Added the in-flight check the markdown probe already has, scoped to the dispatch ref so a manual run on another branch doesn't hold up the scheduled one.

Depends on #16417 for the full fix

Today both checkers share the [link-checker] prefix, so an exact-prefix match still cannot tell them apart — this PR narrows the bug rather than eliminating it. #16417 renames the markdown checker's prefix to [md-link-checker] , which does not satisfy startswith("[link-checker] "). Merged together, the two checkers are fully disambiguated. That PR carries the matching limit and in-flight fixes so the two probes don't drift.

Validation

  • Filter applied to live microsoft/vstest data keeps exactly the genuinely prefixed PRs and drops every false match.
  • Live open-PR count: 1 (suppressed) → 0 (dispatches).
  • Guard branch logic checked against a stubbed gh across open-PR × in-flight combinations (0/0, 1/0, 0/1, 1/1, 0/3).
  • Pagination behaviour verified directly against a repo large enough to exceed one page.
  • Workflow YAML parses; every run: block passes bash -n.

Known defect in the surrounding design, not fixed here

While reviewing this I found that the agent cannot commit the fingerprint it is instructed to update, which breaks the loop both probes depend on. create-pull-request compiles with protect_top_level_dot_folders: true and no protected_dot_folder_excludes. Per gh-aw's checkForTopLevelDotFolders, any path whose first segment starts with . is rejected — including .github/workflows/scripts/known-broken-links.txt — and with protected_files_policy: fallback-to-issue the agent opens an issue instead of a PR. The fingerprint then never updates and the probe re-dispatches every week, which is the exact cost the design set out to avoid.

This affects the already-merged #16413 and the pending #16417 equally, and fixing it means changing the agentic workflow frontmatter plus regenerating its lock. Out of scope for this guard fix — filed as #16422.

The probe's duplicate-run guard narrowed with `gh pr list --search
'"[link-checker]" in:title'` and trusted the resulting count. GitHub discards the
brackets and matches the remaining phrase anywhere in the title, so the query is
really asking "does any open PR title contain the words link checker".

The three queries below return the same 13 PRs in this repo, which is what shows
the brackets contribute nothing; reversing the word order returns none, which is
what shows it is a phrase match rather than loose keywords:

    "[link-checker]"  -> 13
    "link-checker"    -> 13
    "link checker"    -> 13
    "checker link"    -> 0

Only 6 of those 13 carry the prefix. The other 7 are ordinary PRs that merely
mention the link checker, such as microsoft#16104 "Split the link checker workflow into
HTTP and Relative link checker" and microsoft#16093 "Add agentic workflow link-checker".

The failure is silent and points the expensive way. The probe concludes a
checker PR is already open, skips the dispatch, and reports success: the scan
still runs, still finds changed links, and simply declines to act on them, for
as long as the unrelated PR stays open. Right now open PR microsoft#16417 makes the guard
report 1, so the HTTP checker is suppressed today.

Keep the search as cheap server-side narrowing, since it is a superset, and
re-check the real prefix with jq so only an actual checker PR counts.

Verified against microsoft/vstest: the filter keeps exactly the 6 genuinely
prefixed PRs out of the 13 the search returns, and the live open-PR count goes
from 1 (suppressed) to 0 (dispatches). Branch logic re-checked against a stubbed
gh for 0, 1, and 3 open checker PRs; the workflow YAML parses and every run
block passes bash -n.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 27, 2026 14:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the HTTP link-check probe’s duplicate-run guard so it only suppresses dispatch when an open PR actually uses the [link-checker] title prefix, avoiding false suppression caused by GitHub title search behavior.

Changes:

  • Switches the open-PR guard from trusting GitHub’s search hit count to client-side filtering of PR titles via --json title + jq startswith(...).
  • Raises gh pr list --limit to 100 to ensure the client-side prefix filter can inspect enough results.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/http-link-check-probe.yml Outdated
The inline rationale quoted today's repo counts, which drift and would mislead
later readers. Describe the failure mode instead.

Add the in-flight check the markdown probe already has. The open-PR query cannot
see an agent that has started but not yet opened its PR, so a second probe run in
that window dispatched a duplicate. Scope it to the ref being dispatched, so a
manual run on another branch does not hold up the scheduled one.

Verified: guard cases for open-PR and in-flight combinations behave correctly,
YAML parses, and every run block passes bash -n.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

.github/workflows/http-link-check-probe.yml:168

  • The in-flight guard only inspects the most recent 20 runs. If a previous run is stuck (non-completed) but has more than 20 newer completed runs on the same branch, this check can miss it and still dispatch another run. Bumping the limit reduces the chance of truncation hiding an in-flight run.
          IN_FLIGHT=$(gh run list --workflow http-link-checker.lock.yml --branch "$REF_NAME" \
            --limit 20 --json status --jq '[.[] | select(.status != "completed")] | length')

Comment thread .github/workflows/http-link-check-probe.yml
Both guards are correctness checks, so a truncated window is a wrong answer, not
a shorter list. At --limit 100 a busy repo silently returns exactly 100 rows:
the search-backed query on NixOS/nixpkgs returns 215 at --limit 250 but exactly
100 at --limit 100. A prefixed PR pushed out of that window reads as 'none open'
and dispatches a duplicate agent run.

Raise the open-PR query to 500 and the in-flight run query to 200, so a burst of
completed runs cannot hide the one run still in flight either.

gh pages through the API rather than capping at one page, so these are ceilings
on how much we are willing to inspect, not per-request sizes; both stop early at
the real result count, which is 21 open PRs and 11 runs in this repo today.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 15:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@azat-msft
Azat Mukhametshin (azat-msft) merged commit ee6dd4b into microsoft:main Aug 31, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants