docs(flaky-tests): document automatic label application from monitors by samgutentag · Pull Request #651 · trunk-io/docs

samgutentag · 2026-05-13T10:58:36Z

Summary

Replaces the "Coming soon" placeholder in flaky-tests/management/test-labels.md with actual documentation on how monitors can automatically apply labels to test cases
Explains the two monitor action types: Classify test status (flaky/broken) and Apply labels
Documents that action type is set at creation and cannot be changed, with step-by-step instructions for configuring label-applying monitors

Source

trunk2 PR: https://github.com/trunk-io/trunk2/pull/3945

Test plan

Preview in GitBook

…application Monitors can now apply labels to test cases as an alternative to classifying them as flaky or broken. Remove the "coming soon" placeholder in test-labels.md and replace with a concrete how-to for configuring label-applying monitors. Source: trunk-io/trunk2#3945 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

trunk-staging-io · 2026-05-13T10:58:39Z

Merging to main in this repository is managed by Trunk.

To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

claude · 2026-05-13T10:59:10Z

Claude finished @samgutentag's task in 1m 22s —— View job

Claude PR Review

Read changed file (flaky-tests/management/test-labels.md)
Review for typos, grammar, formatting
Check documentation best practices
Post inline review comments
Finalize review

Summary

The PR replaces the "Coming soon" placeholder with clear, well-structured documentation for the new monitor action types. Content is accurate, mirrors the established phrasing pattern from flaky-tests/detection/failure-count-monitor.md ("set at creation and cannot be changed afterward..."), and the new intro sentence on line 7 fits naturally.

No typos, no grammar errors, no broken anchors, no logic issues. The relative link target (#automatic-labeling-from-monitors) is preserved by keeping the same ### Automatic labeling from monitors heading. The "Coming soon" hint was correctly removed.

Inline feedback (all non-blocking)

Numbered list punctuation (line 46–50) — other numbered lists in this directory (managing-detected-flaky-tests.md, failure-count-monitor.md) end each step with a period; these don't. Style consistency nit with a suggestion block.
"(the default behavior)" (line 39) — since Apply labels is also a behavior, (default) parallels better. Optional.
"what monitors set" (line 52) — minor wording polish for parallelism with "apply" verbs used earlier. Optional.

Verdict

✅ Approving from a docs-review standpoint. The one style consistency nit on the numbered list is worth picking up, but nothing here blocks merging.
· branch: sam-gutentag/monitor-action-types-and-auto-labeling

claude · 2026-05-13T11:00:31Z

+1. Go to **Flaky Tests** > select your repository > **Monitors**
+2. Click **Add** next to the monitor type you want to create
+3. In the creation form, select **Apply labels** as the action type
+4. Choose one or more labels to apply when the monitor activates
+5. Complete the rest of the monitor configuration and save


Style consistency — numbered list punctuation.

Other numbered lists in this directory (e.g., flaky-tests/management/managing-detected-flaky-tests.md and flaky-tests/detection/failure-count-monitor.md) end each step with a period. These steps don't, which makes the list look inconsistent with the rest of the docs.

Suggested change

1. Go to **Flaky Tests** > select your repository > **Monitors**

2. Click **Add** next to the monitor type you want to create

3. In the creation form, select **Apply labels** as the action type

4. Choose one or more labels to apply when the monitor activates

5. Complete the rest of the monitor configuration and save

1. Go to **Flaky Tests** > select your repository > **Monitors**.

2. Click **Add** next to the monitor type you want to create.

3. In the creation form, select **Apply labels** as the action type.

4. Choose one or more labels to apply when the monitor activates.

5. Complete the rest of the monitor configuration and save.

Fix this →

claude · 2026-05-13T11:00:33Z

+
+When creating a failure count or failure rate monitor, you choose the **action type**:
+
+* **Classify test status** — the monitor marks tests as flaky or broken (the default behavior)


Minor: "(the default behavior)" — since the parallel bullet Apply labels also describes a behavior, "default behavior" reads slightly awkwardly. Consider just "(default)" for parallelism, e.g.:

Suggested change

* **Classify test status** — the monitor marks tests as flaky or broken (the default behavior)

* **Classify test status** — the monitor marks tests as flaky or broken (default)

Non-blocking.

claude · 2026-05-13T11:00:35Z

+4. Choose one or more labels to apply when the monitor activates
+5. Complete the rest of the monitor configuration and save
+
+Tests that the monitor flags will have the selected labels applied automatically. You can still apply or remove labels manually on top of what monitors set.


Optional wording polish: "what monitors set" is a bit terse and uses passive voice. Consider:

Suggested change

Tests that the monitor flags will have the selected labels applied automatically. You can still apply or remove labels manually on top of what monitors set.

When the monitor flags a test, the selected labels are applied automatically. You can still apply or remove labels manually on top of what monitors apply.

Or keep the passive but use "apply" for parallelism with the rest of the section: "on top of what monitors apply" rather than "set". Non-blocking.

github-actions

Auto-approved: Claude code review passed.

samgutentag · 2026-05-13T17:34:42Z

Verification status (2026-05-13): pending

Eng merged but flag off in prod. Hold off.

Eng PR: trunk-io/trunk2#3945 (merged 2026-05-12), with predecessor trunk-io/trunk2#3774 (merged 2026-05-04)
Flag: showLabelActions (LaunchDarkly, frontend-web-2 project)
Signals checked:
- e2e default: true (test-only, does not confirm prod)
- Follow-up PRs in trunk2 after 2026-05-12: none found
- Slack signals after merge: none. Earlier LD bot messages (2026-04-29 to 04-30) are pre-merge background and excluded by the recency rule.
- Legacy code: showLabelActions still actively gates the new UI in monitor-actions-section.tsx and monitor/page.tsx on main

Hold until the flag flips to 100% in prod (or a follow-up PR removes the gate). Re-run /verify-docs-pr 651 after rollout.

claude · 2026-05-13T18:09:01Z

Claude finished @samgutentag's task in 58s —— View job

Claude PR Review — re-review on ready_for_review

Read changed file (flaky-tests/management/test-labels.md)
Check git diff vs origin/main
Verify style consistency claims against sibling docs
Compare against prior review (2026-05-13 11:00 UTC) — same file SHA, no new content to re-evaluate

Summary

No new changes since the prior review at 2026-05-13 11:00 UTC (file SHA 0f151ca — unchanged). The three earlier inline comments still stand and are all non-blocking style nits:

Numbered list trailing periods (lines 46–50) — confirmed: managing-detected-flaky-tests.md:28-30 and failure-count-monitor.md:48-53 both terminate each step with a period. Worth fixing for consistency.
"(the default behavior)" (line 39) — (default) reads more parallel against the Apply labels bullet. Optional.
"what monitors set" (line 52) — minor passive-voice/parallelism polish (apply rather than set). Optional.

No typos, grammar errors, broken links, or structural issues. Anchor #automatic-labeling-from-monitors is preserved by the unchanged heading. The intro sentence on line 7 reads cleanly.

Verification status

Per @samgutentag's 2026-05-13 17:34 UTC comment, the eng PR (trunk-io/trunk2#3945) merged 2026-05-12 but showLabelActions is still gating the UI in prod. Docs content is correct on its own merits, but don't merge until the flag flips to 100% — otherwise users will read about a UI they can't see.

Verdict

✅ Approving from a docs-review standpoint. The numbered-list period nit is worth picking up before merge; the other two are optional polish. Hold the merge for the flag rollout, not for the review.
· branch: sam-gutentag/monitor-action-types-and-auto-labeling

github-actions

Auto-approved: Claude code review passed.

samgutentag · 2026-05-13T19:45:09Z

superseded by #656

claude Bot reviewed May 13, 2026

View reviewed changes

github-actions Bot approved these changes May 13, 2026

View reviewed changes

samgutentag changed the title ~~docs(flaky-tests): document monitor action types and automatic label application~~ [feature not live] docs(flaky-tests): document monitor action types and automatic label application May 13, 2026

samgutentag changed the title ~~[feature not live] docs(flaky-tests): document monitor action types and automatic label application~~ docs(flaky-tests): document automatic label application from monitors May 13, 2026

samgutentag mentioned this pull request May 13, 2026

docs(flaky-tests): document monitor action type selection for failure rate and failure count monitors #655

Closed

3 tasks

samgutentag marked this pull request as ready for review May 13, 2026 18:08

github-actions Bot approved these changes May 13, 2026

View reviewed changes

samgutentag closed this May 13, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs(flaky-tests): document automatic label application from monitors#651

docs(flaky-tests): document automatic label application from monitors#651
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/monitor-action-types-and-auto-labeling

samgutentag commented May 13, 2026

Uh oh!

trunk-staging-io Bot commented May 13, 2026

Uh oh!

claude Bot commented May 13, 2026 •

edited

Loading

Uh oh!

claude Bot May 13, 2026

Uh oh!

claude Bot May 13, 2026

Uh oh!

claude Bot May 13, 2026

Uh oh!

github-actions Bot left a comment

Uh oh!

samgutentag commented May 13, 2026

Uh oh!

claude Bot commented May 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot left a comment

Uh oh!

samgutentag commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants


		When creating a failure count or failure rate monitor, you choose the action type:

		* Classify test status — the monitor marks tests as flaky or broken (the default behavior)

	Tests that the monitor flags will have the selected labels applied automatically. You can still apply or remove labels manually on top of what monitors set.
	When the monitor flags a test, the selected labels are applied automatically. You can still apply or remove labels manually on top of what monitors apply.

Conversation

samgutentag commented May 13, 2026

Summary

Source

Test plan

Uh oh!

trunk-staging-io Bot commented May 13, 2026

Uh oh!

claude Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Claude PR Review

Summary

Inline feedback (all non-blocking)

Verdict

Uh oh!

claude Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

samgutentag commented May 13, 2026

Uh oh!

claude Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification status

Verdict

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

samgutentag commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

claude Bot commented May 13, 2026 •

edited

Loading

claude Bot commented May 13, 2026 •

edited

Loading