Skip to content

Keep plugin hot paths off direct database work#1862

Draft
Copilot wants to merge 7 commits intomainfrom
copilot/fix-mau-bandwidth-chart-issue
Draft

Keep plugin hot paths off direct database work#1862
Copilot wants to merge 7 commits intomainfrom
copilot/fix-mau-bandwidth-chart-issue

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 25, 2026

Summary (AI generated)

This PR fixes a regression from the dashboard refresh work that moved cron-stat scheduling onto live plugin traffic by calling a service-role DB RPC from stats ingestion. It removes that request-path queueing so /stats and bandwidth tracking stop touching Postgres just to schedule chart refresh work.

It also adds an explicit repository guardrail in AGENTS.md so future changes do not reintroduce direct database work on performance-sensitive plugin endpoints.

  • Root cause

    • createStatsMau() and createStatsBandwidth() were calling queue_cron_stat_app_for_app(...) from the live request path.
    • That pushed primary-database work into high-volume plugin endpoints.
  • Change

    • Remove direct cron queue RPC calls from supabase/functions/_backend/utils/stats.ts.
    • Keep live stats helpers limited to recording usage/log data.
    • Leave cron refresh scheduling on the async SQL sweep path (process_cron_stats_jobs()), where it belongs.
    • Update AGENTS.md to explicitly forbid direct primary-DB work, service-role RPCs, or queue writes from /updates, /stats, and /channel_self request paths.
  • Regression coverage

    • Update tests/process-cron-stats-jobs.test.ts to assert:
      • process_cron_stats_jobs() still queues active apps.
      • live /stats traffic does not enqueue cron work directly.
// Before
await supabaseAdmin(c).rpc('queue_cron_stat_app_for_app', { p_app_id: app_id })

// After
return backgroundTask(c, trackingJob.then(() => undefined))

Test plan (AI generated)

  • Verify the cron sweep still queues active apps via process_cron_stats_jobs().
  • Verify live /stats traffic no longer creates cron_stat_app queue entries.
  • Verify AGENTS.md explicitly documents that /updates, /stats, and /channel_self must not add direct primary-database work on the live request path.
  • Run the existing backend and Cloudflare backend test suites covering the affected paths.
  • Confirm CI passed for lint, typecheck, backend tests, and Cloudflare backend tests on the final branch state.

Screenshots (AI generated)

  • None. Backend and internal documentation only.

Checklist (AI generated)

  • My code follows the code style of this project and passes
    bun run lint:backend && bun run lint.
  • My change requires a change to the documentation.
  • I have updated the documentation
    accordingly.
  • My change has adequate E2E test coverage.
  • I have tested my code manually, and I have provided steps how to reproduce
    my tests
Original prompt

We have a fucking problem because you're an idiot. We made a pull request to fix the MAU chart and bandwidth chart, and the modif you did is to call the fucking database directly from this endpoint, where this endpoint is fucking sensible and called by millions of devices a second. It cannot call the primary database unless there is a very specific use case that cannot be avoided, and even then, it shouldn't do it in any way most of the time. Now Capgo is fucking destroyed and cannot fucking work. So fucking fix this shit! Explain to me what you have done in the pull request that has been merged this morning at 6:00 a.m. because this is fucking broken, you broke everything!


📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

@riderx
Copy link
Copy Markdown
Member

riderx commented Mar 25, 2026

@copilot On top of that please fucking idiot wrote into the agent's agent.md file that it is very important to not put into the update_stats channel anything that is going to directly the database. It is not allowed; it is forbidden; it is the worst thing to do in a fucking universe. Tell it that it's very important and that it shouldn't never be done that way, never ever. You have to find an alternative that is not going directly to the database because these endpoints are sensible in terms of performance and call B my second. Rot it.

Copilot AI changed the title [WIP] Fix direct database call in MAU and bandwidth charts Keep cron stats queueing off the live stats hot path Mar 25, 2026
Copilot AI requested a review from riderx March 25, 2026 14:25
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 25, 2026

@copilot On top of that please fucking idiot wrote into the agent's agent.md file that it is very important to not put into the update_stats channel anything that is going to directly the database. It...

Added the guardrail to AGENTS.md in a11e513. It now explicitly says /updates, /stats, and /channel_self must never add direct primary-DB work, service-role RPCs, or queue writes on the live request path, and that async sweeps / existing analytics pipelines must be used instead.

Copilot AI changed the title Keep cron stats queueing off the live stats hot path Keep plugin hot paths off direct database work Mar 25, 2026
@riderx
Copy link
Copy Markdown
Member

riderx commented Mar 26, 2026

Resolved the merge conflicts in AGENTS.md and supabase/functions/_backend/utils/stats.ts when updating the branch with origin/main. I unified the hot-path guidance so the /updates, /stats, and /channel_self sections include both the replica-only warning plus the explicit call-outs about not queueing primary DB work, and I kept the stats helpers using the Promise.all/backgroundTask flow while leaving the old queue cron helper as a comment for future reference. Branch now contains the latest main commits.

@riderx
Copy link
Copy Markdown
Member

riderx commented Mar 27, 2026

Merged origin/main manually because the hot-path guidance in AGENTS.md conflicted. Kept the hot-path warning from the branch, then added the newer phrasing from main so the section now says these endpoints may not perform direct primary DB work, they may not enqueue writes via backgroundTask, and any unavoidable primary writes need minimal scope and inline justification plus extra review.

…idth-chart-issue

# Conflicts:
#	tests/process-cron-stats-jobs.test.ts
@riderx
Copy link
Copy Markdown
Member

riderx commented Mar 28, 2026

Merged origin/main into this branch to keep it current. There was a conflict in tests/process-cron-stats-jobs.test.ts around the newly-added live /stats test, so I manually reattached that test after the prior cron_stat_app verification and kept the existing expectations intact before pushing the result.

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Mar 28, 2026

Merging this PR will not alter performance

✅ 28 untouched benchmarks


Comparing copilot/fix-mau-bandwidth-chart-issue (86b6171) with main (a843519)

Open in CodSpeed

@sonarqubecloud
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants