feat: conferences summary endpoint + indexes (Phase 0 + Phase 1 of #20) by Boanerges1996 · Pull Request #24 · peermetrics/api

Boanerges1996 · 2026-04-16T18:18:37Z

Phase 0 + Phase 1 of #20 — database indexes plus the server-side aggregation endpoint for the Conferences chart.

Problem

The dashboard graphs tab currently downloads all raw conferences into the browser and aggregates them with JavaScript. On production: 42,177 conferences / 21.7 MB / ~13 seconds per dashboard load, on a trajectory to exceed the 30s gunicorn timeout and start returning 504s.

This PR

Phase 0 — Indexes (migration 0004)

Adds the indexes required for the summary query to scale on production. Without these, aggregation queries still do full table scans on 42K+ rows.

Index	Purpose
`conference(app, -created_at)`	Summary endpoint filter + GROUP BY
`issue(conference, type)`	Exists subqueries for has_error / has_warning
`app_genericevent(app, -created_at)`	Future event-based summaries + cleanup queries

Phase 1 — Summary endpoint

New GET /v1/conferences/summary?appId=<id>&created_at_gte=<iso>&created_at_lte=<iso> returning daily counts bucketed by status:

{
  "data": [
    {"date": "2026-04-07", "success": 416, "warning": 0, "error": 0, "ongoing": 0, "total": 416},
    {"date": "2026-04-08", "success": 148, "warning": 0, "error": 1, "ongoing": 0, "total": 149}
  ]
}

Measured on local test data (2,573 conferences)

	Size	Time
Full list (current)	1,139,816 bytes	~350ms
Summary	854 bytes	<200ms
Size reduction	1,335× smaller

On production (42K conferences, 21.7 MB) the reduction will be much larger because the summary scales with number of days (~30 rows) regardless of conference count.

Implementation notes

CASE WHEN expression classifies each conference into one of ongoing | error | warning | success (mutually exclusive stacks the chart renders).
Postgres rejects correlated EXISTS subqueries inside COUNT(...) FILTER clauses, so we annotate the classification first and group by (day, status) afterwards.
Accepts both native Python ISO format and the Z suffix that JavaScript's toISOString() produces.

Scope

This PR: indexes + the summary endpoint only. The existing /v1/conferences list endpoint is unchanged and still used by the Conference list tab and other charts.
Web-side migration of the Conferences chart: feat: conferences chart uses summary endpoint (Phase 1 of #20) web#25.
Remaining charts (duration, issues, participants, connections, sessions) will migrate in follow-up PRs as described in Add server-side aggregation endpoints for dashboard charts #20.

Test plan

Migration applies cleanly; indexes appear in pg_indexes
Curl the endpoint with valid params — returns 200 with daily rows
Curl with bad ISO string — returns 400
Curl with no appId — returns 400
Handles Z suffix from JavaScript toISOString
Data matches client-side aggregation over the same dataset

Returns conference counts grouped by day, bucketed into success/warning/error/ongoing. This replaces the pattern of downloading all conferences to the browser and aggregating client-side. Observed benefit on local test data (2,573 conferences, 1.1 MB response): - Full list: 1,139,816 bytes - Summary: 854 bytes (1335x smaller) Supports appId, created_at_gte, created_at_lte query params. Handles both Python 3.8 native ISO format and the Z suffix that JavaScript toISOString produces. Implementation uses a CASE expression to classify each conference as ongoing/error/warning/success (mutually exclusive stacks the chart renders), then groups by (day, status). Works around Postgres's restriction that correlated subqueries (Exists) aren't allowed inside COUNT(...) FILTER clauses. Phase 1 of peermetrics#20.

…g client-side The conferences chart now calls the new /v1/conferences/summary endpoint (peermetrics/api#24) and renders from pre-computed daily counts. No more downloading all conferences to the browser just to compute bar chart numbers. Click-to-detail modal: when the user clicks a bar, fetch just that day's conferences via the paginated list (?created_at_gte=<day-start>& created_at_lte=<day-end>&limit=50) and filter by the clicked status client-side. Small result set, fast response. The parent app.vue still fetches the full conferences list for the remaining charts (duration, participants, issues) until their phases.

Prerequisite for the summary endpoint (Phase 1) to scale on production. Without these, the aggregation query still runs a full table scan on 42K+ conferences. - conference(app, -created_at): filter + order for summary endpoint - issue(conference, type): speeds up the Exists subqueries that classify conferences as has_error / has_warning - app_genericevent(app, -created_at): future event summary endpoints and cleanup queries Per the phased plan in peermetrics#20 and peermetrics#15.

agonza1 · 2026-04-16T21:15:26Z

@codex

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c87cd87078

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-04-16T21:21:02Z

+        migrations.AddIndex(
+            model_name='conference',
+            index=models.Index(
+                fields=['app', '-created_at'],
+                name='idx_conf_app_created',


Declare dashboard indexes in model metadata

These indexes are introduced only through AddIndex operations in a migration, but the corresponding models don’t define matching Meta.indexes. That creates a migration-state mismatch where future makemigrations runs can generate RemoveIndex operations for these indexes, making it easy to accidentally drop performance-critical indexes during unrelated schema work.

Useful? React with 👍 / 👎.

Addresses review feedback. The indexes were only defined in the migration's AddIndex operations; the models didn't declare them in Meta.indexes. That created a state mismatch where future makemigrations runs would generate RemoveIndex operations for these performance-critical indexes during unrelated schema work. Verified: `python manage.py makemigrations --dry-run` now reports "No changes detected" after the model updates.

* feat: conferences chart fetches its own summary instead of aggregating client-side The conferences chart now calls the new /v1/conferences/summary endpoint (peermetrics/api#24) and renders from pre-computed daily counts. No more downloading all conferences to the browser just to compute bar chart numbers. Click-to-detail modal: when the user clicks a bar, fetch just that day's conferences via the paginated list (?created_at_gte=<day-start>& created_at_lte=<day-end>&limit=50) and filter by the clicked status client-side. Small result set, fast response. The parent app.vue still fetches the full conferences list for the remaining charts (duration, participants, issues) until their phases. * fix: address Codex review on conferences chart Two P2 issues flagged: 1. Year-boundary bug: moment(e.xValue, 'MM/DD').year(now) resolves '12/31' to the current year even in early January, so clicking an end-of-year bar would query the wrong year. Fixed by tracking ISO dates (YYYY-MM-DD) in a parallel array and resolving the clicked bar's date via its label index rather than re-parsing the display string. 2. Modal showed only 50 conferences even when the chart bar said 400+. Now paginates through all pages for the clicked day via the existing /v1/conferences limit/offset API (200 per page, uses the returned `count` field to know when to stop). The summary lookup map is also now keyed by ISO date instead of MM/DD, which was another latent bug when the date window crossed a year.

Boanerges1996 mentioned this pull request Apr 16, 2026

feat: conferences chart uses summary endpoint (Phase 1 of #20) peermetrics/web#25

Merged

4 tasks

Boanerges1996 changed the title ~~feat: add GET /v1/conferences/summary endpoint for dashboard charts~~ feat: conferences summary endpoint + indexes (Phase 0 + Phase 1 of #20) Apr 16, 2026

chatgpt-codex-connector Bot reviewed Apr 16, 2026

View reviewed changes

agonza1 merged commit d9f3857 into peermetrics:master Apr 17, 2026

This was referenced Apr 20, 2026

fix: unblock conferences chart — remove v-if waiting on full conferences list peermetrics/web#26

Merged

feat: duration/participants/issues/gum summary endpoints (Phase 2 + 3 of #20) #26

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: conferences summary endpoint + indexes (Phase 0 + Phase 1 of #20)#24

feat: conferences summary endpoint + indexes (Phase 0 + Phase 1 of #20)#24
agonza1 merged 3 commits intopeermetrics:masterfrom
Boanerges1996:feat/conferences-summary-endpoint

Boanerges1996 commented Apr 16, 2026 •

edited

Loading

Uh oh!

agonza1 commented Apr 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Apr 16, 2026

Uh oh!

Boanerges1996 Apr 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Boanerges1996 commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

This PR

Phase 0 — Indexes (migration 0004)

Phase 1 — Summary endpoint

Measured on local test data (2,573 conferences)

Implementation notes

Scope

Test plan

Uh oh!

agonza1 commented Apr 16, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Boanerges1996 Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Boanerges1996 commented Apr 16, 2026 •

edited

Loading