Send Android Toolbar and Homepage, IP Protection, and Sharing bugs for automatic triage - #2996
Open
msujaws wants to merge 3 commits into
Open
Send Android Toolbar and Homepage, IP Protection, and Sharing bugs for automatic triage#2996msujaws wants to merge 3 commits into
msujaws wants to merge 3 commits into
Conversation
…c triage, now that all three have channels Three pairs join `TRIAGED_COMPONENTS`. `get_bz_params` loops the tuple to build one AND group per pair, so the query needs no change: I ran the generated chart against BMO over 90 days and got 263 open defects against the eight pairs, up from 178, split 35 Homepage, 32 IP Protection and 18 Toolbar. No cross pairing appeared, and none can -- BMO has no `Firefox :: Homepage` and no `Firefox :: Toolbar` (the desktop component is `Toolbars and Customization`), and `IP Protection` exists only under `Firefox`. The AND groups are what keep that true if somebody creates one. `max_triggers` stays at 3. Staff-filed volume goes from about 120 to 168 per 90 days, which is 1.87/day against the 72/day an hourly run with a cap of 3 allows; four filings in these components inside one hour is a ~10^-6 event, and a burst drains over successive runs inside the 24-hour `days_lookup` window with `left_for_next_run` reporting the backlog. What actually changes is spend, roughly 40 agent runs a month to 56. The cap also bounds how many unreviewed Bugzilla comments a bad prompt change can produce in an hour, and three components are about to start posting unattended, so leaving it at 3 is the positive choice rather than the absence of a reason to raise it. Requires the matching bugbug changes to be deployed first. The comment above `TRIAGED_COMPONENTS` now says so, since until now that constraint lived only in commit messages: `channel_for` fails closed on a component missing from the agent's `TRIAGE_SCOPE`, which silences the notification but not the run, so the analysis and the severity change land with nobody told. Nothing checks the two lists agree and nothing can -- they are in separate repos. Because scripts/cron_common_start.sh checks out `releases/latest`, this commit is inert on master; the release cut is the switch, so the gate is "agent rolled out before the release", not merge order. No test changes; the 27 tests derive from `TRIAGED_COMPONENTS` and pass unchanged. I could not run `--dryrun`, since it needs the gitignored `configs/people.json`.
…has a channel and knows the share sheet is in widget/ One pair joins `TRIAGED_COMPONENTS`. Volume is high for a component this narrow: 38 open defects filed in the last 90 days, 34 of them by staff, against 22 for IP Protection and 90 for New Tab Page. `max_triggers` stays at 3. Staff-filed volume across the nine pairs goes to about 202 per 90 days, or 2.24/day, against the 72/day an hourly run with a cap of 3 allows. The cap still only binds on four filings in these components inside one hour, and `left_for_next_run` reports the backlog when it does. Requires the matching hackbot change to be deployed first, and `#content-sharing-automation` to exist and be public before that. `channel_for` fails closed, so a missing entry or a wrong channel name silences the notification but not the run. No test changes; the 27 tests derive from `TRIAGED_COMPONENTS`.
…ngs drains in fewer hours `max_triggers` is a per-run cap and the rule runs hourly, so this raises the ceiling from 72 to 120 runs a day against staff-filed demand of roughly 2.24/day across the nine components now in scope. Steady state is unaffected -- the cap has never bound at that rate. What changes is how fast a burst clears: ten bugs from one QA session drained over four hourly runs at 3 and clears in two at 5, and each run held back was reported in the email as `left_for_next_run` rather than lost. The cost is the other thing this cap bounds. It is also the limit on how many unreviewed Bugzilla comments a bad prompt or ruleset change can produce in one hour before anyone notices, so that worst case goes from 3 to 5. No test change: both cap tests set `max_triggers` explicitly rather than reading the config, and 27 pass.
suhaibmujahid
approved these changes
Aug 17, 2026
Member
|
Will merge after deploying the agent. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four components join the set bugbot sends to hackbot's
frontend-triageagent:Firefox for Android :: Toolbar,Firefox for Android :: Homepage,Firefox :: IP Protection, andFirefox :: Sharing. Each has a Slack channel on the agent side, and each of those channels exists and is public.On this side that is three lines in
TRIAGED_COMPONENTS. #2978 already did the generalization that makes it cheap:get_bz_paramsloops the tuple to build one AND group per pair inside a top-level OR, and the 27 tests derive from the tuple, so neither the query nor a test needs touching.Depends on mozilla/bugbug#6617
That must be rolled out first.
channel_foron the agent side fails closed on a component missing from its registry, which silences the Slack notification but not the run — the analysis comment and the severity change still apply, so the component would get unattended triage with nobody told.The gate is the deploy, not the merge.
scripts/cron_common_start.sh:20checks out$(… releases/latest … .tag_name), so this PR is inert on master and the release cut is the switch. Merging it early is harmless; cutting a release before #6617 is live is not.Because that constraint has until now lived only in commit messages (8bbbc55, #2991), it is written into the comment above
TRIAGED_COMPONENTSin this PR. Nothing checks that the two lists agree and nothing can — they are in separate repos and cannot see each other, which is deliberate: the comment says the reach lives in code so that widening it takes a code review, and a runtime fetch would let a bugbug commit widen what bugbot sends with no review here.The query does not widen accidentally
I ran the generated boolean chart against BMO over 90 days: 263 open defects across the eight pairs before Sharing, up from 178, and the per-component counts match individual queries exactly. No cross pairing appeared, and none is currently possible — BMO has no
Firefox :: Homepageand noFirefox :: Toolbar(the desktop component isToolbars and Customization), andIP ProtectionandSharingexist only underFirefox. That is today's state rather than a guarantee, which is exactly whyget_bz_paramsbuilds per-pair AND groups instead of passing flatproduct/componentlists.Staff-filed volume over 90 days, using a reporter-contains-
@mozilla.comapproximation (it reproduced the "90 for New Tab Page" figure in 8bbbc55):Firefox :: SharingFirefox for Android :: HomepageFirefox :: IP ProtectionFirefox for Android :: Toolbarmax_triggers3 → 5This is a per-run cap and the rule runs hourly, so the ceiling goes from 72 to 120 runs a day against staff-filed demand of roughly 2.24/day across the nine components now in scope. Steady state is unaffected — the cap has never bound at that rate. What changes is burst drain: ten bugs from one QA session cleared over four hourly runs at 3 and clears in two at 5, and every run that held bugs back already reported them in the email as
left_for_next_runrather than losing them.The cost is the other thing this cap bounds. It is also the limit on how many unreviewed Bugzilla comments a bad prompt or ruleset change can produce in one hour before anyone notices, so that worst case goes from 3 to 5.
Testing
uv run pytest tests/rules/test_frontend_triage.py— 27 pass, unchanged.uv run pytest tests/— 75 pass.No test changes were needed, which is the point of #2978's design:
test_queries_every_triaged_componentcompares the reconstructed chart tolist(TRIAGED_COMPONENTS),test_ors_the_component_groups_and_ands_within_eachderives its expected length from it, andtest_pairs_a_component_with_its_own_productis the regression guard for the cross-pairing widening described above. Both cap tests setmax_triggersexplicitly rather than readingconfigs/rules.json, so the 3 → 5 change does not touch them.I could not run
python -m bugbot.rules.frontend_triage --dryrun: it needs the gitignoredconfigs/people.json, which is absent here. The BMO chart check above is what stands in for it.Scope
Nothing else in this repo is keyed by component.
templates/frontend_triage.htmlrendersProduct :: Componentgenerically becausehas_product_component()returns True, andscripts/check_rules_on_wiki.pystill exempts this rule while it is piloting.configs/rules.jsonchanges only formax_triggers.