-
Notifications
You must be signed in to change notification settings - Fork 327
[doc-healer] DDUw improvement: explicitly handle new-file guide creation requests in Step 1b #24741
Description
Background
Analyzed 20 documentation issues closed between 2026-03-29 and 2026-04-05. All gaps are now fixed, but one recurrent pattern was identified: DDUw silently skips open documentation issues that request creation of a new guide file rather than updating an existing page.
Instance Found
Issue #23958 — docs: add guide for consuming audit reports with an agent (filed 2026-04-01, closed 2026-04-04)
- DDUw ran on 2026-04-02 (PR [docs] Update documentation for 2026-04-02 features #24170), 2026-04-03 (PR [docs] Update documentation for features from 2026-04-03 #24377), and 2026-04-04 (PR [docs] Update documentation for features from 2026-04-04 #24574)
- Issue docs: add guide for consuming audit reports with an agent #23958 is completely absent from all three DDUw PRs — no mention, no skip note, no "already covered" annotation
- A human had to manually invoke Copilot to create the file via docs: add guide for consuming audit reports with agents #24454 (merged 2026-04-04)
- A closely related issue, docs: add audit command reference page #23957 (add audit reference page), was addressed by DDUw PR [docs] Update documentation for 2026-04-02 features #24170 in the same run
- The distinction: docs: add audit command reference page #23957 requested updating reference docs; docs: add guide for consuming audit reports with an agent #23958 requested creating a brand-new 127-line how-to guide
Root Cause
DDUw's Step 1b instructs: "Check the referenced documentation file to verify the gap still exists." This works well for issues that point to a known existing file. But when the requested file does not yet exist, there is no file to check — DDUw appears to interpret the absence of a file path to check as "gap is unclear, skip."
There is no explicit branch in Step 1b for the case:
Issue body says "create
docs/.../foo.md" anddocs/.../foo.mddoes not exist yet.
Which DDUw Step Failed
Step 1b — Check Open Documentation Issues, specifically the verification sub-step:
"Check the referenced documentation file to verify the gap still exists."
When the gap requires file creation (not file editing), this check produces a false negative (file not found → issue skipped) instead of a true positive (file not found → gap confirmed).
Proposed Fix
Add an explicit branch in Step 1b after the "check the referenced documentation file" step:
### 1b. Check Open Documentation Issues
For each open issue:
1. Read the issue body to understand the described gap.
2. Check the referenced documentation file to verify the gap still exists.
- **If the issue references an existing file**: confirm the content is missing or incorrect.
- **If the issue references a file path that does not yet exist** (e.g., body says "Create `docs/guides/foo.md`"): treat this as a **confirmed new-file gap** and proceed to Step 5.
3. If confirmed, include a fix in this run's PR and reference the issue with `Closes #NNN`.
4. If the gap is already fixed (file exists and contains the described content), note it and skip.Also add a skip-log requirement: when DDUw chooses not to address an open documentation issue, it MUST add a note to the PR description:
### Skipped Issues
- #NNN — [title]: [reason for skip, e.g., "requires structural nav changes beyond docs scope"]
This prevents silent drops and makes gaps visible even when DDUw cannot fully address them.
Why This Matters
Between 2026-04-01 and 2026-04-04, the audit guide gap sat unaddressed for 3 DDUw runs. A human had to intervene. Making the skip-or-act decision explicit in Step 1b would either surface the guide creation earlier or at minimum flag it visibly in a DDUw PR for human follow-up.
References
- Issue that was missed: docs: add guide for consuming audit reports with an agent #23958
- Manual fix PR: docs: add guide for consuming audit reports with agents #24454
- DDUw PRs that ran while issue was open: [docs] Update documentation for 2026-04-02 features #24170, [docs] Update documentation for features from 2026-04-03 #24377, [docs] Update documentation for features from 2026-04-04 #24574
- Healer workflow run: §24002331082
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #20207
search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | noneGenerated by Daily Documentation Healer · ● 312.5K · ◷
- expires on Apr 8, 2026, 1:28 PM UTC