Analytics: add per-total-score quartile sparklines for each question by Copilot · Pull Request #1449 · inducer/relate

Copilot · 2026-03-24T17:01:56Z

Original prompt

This section details on the original issue you should resolve

<issue_title>Idea: Analytics should also show per-total-score quartile information for each question</issue_title>
<issue_description></issue_description>

Comments on the Issue (you are @copilot in this section)
@inducer That is, for each question, there should be additional, smaller graphs (using https://github.com/maxkostinevich/sparklines.js) showing the score percentage for students in each of the quartiles with respect to their overall score.

Fixes Idea: Analytics should also show per-total-score quartile information for each question #840

⌨️ Start Copilot coding agent tasks without leaving your editor — available in VS Code, Visual Studio, JetBrains IDEs and Eclipse.

Copilot

Pull request overview

Adds per-question “score by overall-session quartile” visualization to the flow analytics page, letting instructors see whether questions discriminate between stronger and weaker students.

Changes:

Backend: compute session quartiles from overall session score and include per-quartile average correctness in PageAnswerStats.
Frontend: add a new JS bundle to render 4-point sparklines from per-quartile data.
Template/build: load the new bundle on analytics-flow.html and add data attributes; wire bundle into Rollup/CI; add sparklines dependency.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`course/analytics.py`	Computes session→quartile mapping and accumulates per-quartile correctness into `PageAnswerStats`.
`course/templates/course/analytics-flow.html`	Loads the new bundle and adds a sparkline element per question (layout refactor to table).
`frontend/js/analytics.js`	Renders sparklines from embedded quartile correctness JSON.
`rollup.config.mjs`	Adds a new Rollup bundle output (`bundle-analysis.js`).
`package.json` / `package-lock.json`	Adds `sparklines` dependency and build script.
`.ci/run-tests-for-ci.sh`	Adds the new bundle to the CI “touched staticfiles” list for `collectstatic`.
`tests/test_analytics.py`	Adds tests for `quartile_correctness_json` serialization.
`biome.json`	Reformatting only.
`.basedpyright/baseline.json`	Updates type-check baseline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-26T21:45:06Z

+def compute_session_quartile_map(
+        pctx: CoursePageContext,
+        flow_id: str) -> dict[int, int]:
+    """Return a mapping from session ID to quartile index (0 = Q1, ..., 3 = Q4).
+
+    Quartiles are assigned based on each session's percentage score, using only
+    completed sessions for participants with grade-statistics permission.
+    """
+    sessions = FlowSession.objects.filter(
+            course=pctx.course,
+            flow_id=flow_id,
+            in_progress=False,
+            participation__roles__permissions__permission=(
+                PPerm.included_in_grade_statistics))
+
+    session_scores: list[tuple[int, float]] = []
+    for session in sessions:
+        pperc = session.points_percentage()
+        if pperc is not None:
+            session_scores.append((session.id, float(pperc)))
+
+    if not session_scores:
+        return {}
+
+    session_scores.sort(key=operator.itemgetter(1))
+    n = len(session_scores)
+    return {
+        sid: min(NUM_QUARTILES - 1, int(i * NUM_QUARTILES / n))
+        for i, (sid, _score) in enumerate(session_scores)
+    }


New behavior in compute_session_quartile_map (quartile assignment logic) is not covered by tests. The added tests only validate JSON serialization, but don't verify that sessions are bucketed into quartiles correctly (including edge cases like small N). Please add a unit test that creates sessions with known points/max_points and asserts the resulting quartile indices.

@copilot apply changes based on this feedback, i.e. add unit tests for compute_session_quartile_map

Added ComputeSessionQuartileMapTest in commit 3fc40eb with 8 unit tests covering:

Empty session set → empty map

Sessions without points excluded

In-progress sessions excluded

Single session → Q1 (index 0)

4 sessions correctly assigned to Q1–Q4

Small-N edge cases (n=2 and n=3)

Sorting by score regardless of insertion order

…lytics - Add compute_session_quartile_map() to analytics.py to assign sessions to Q1-Q4 quartiles based on total score percentage - Add quartile_correctness_list field and quartile_correctness_json property to PageAnswerStats dataclass - Update make_page_answer_stats_list() to compute per-quartile average correctness for each page - Add frontend/js/analytics.js using sparklines npm package to render per-quartile sparklines for each question - Add bundle-analytics to rollup.config.mjs - Update analytics-flow.html to show sparkline charts and load analytics bundle - Install sparklines npm package - Add PageAnswerStatsTest tests for quartile_correctness_json" Co-authored-by: inducer <352067+inducer@users.noreply.github.com> Agent-Logs-Url: https://github.com/inducer/relate/sessions/90d505da-f8b0-4ecc-8317-d826bd923f6a Add unit tests for compute_session_quartile_map Tests cover: empty set, sessions without points, in-progress sessions, single-session, 4 sessions (one per quartile), small-N edge cases (2 and 3 sessions), and ordering by score regardless of insertion order. Agent-Logs-Url: https://github.com/inducer/relate/sessions/890b63ca-7ebb-4571-ab4c-6844838e9121 Co-authored-by: inducer <352067+inducer@users.noreply.github.com> Translations Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Add distinct Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Fix mid-file import Fix HTML

Copilot AI assigned Copilot and inducer Mar 24, 2026

Copilot started work on behalf of inducer March 24, 2026 17:02 View session

Copilot AI changed the title ~~[WIP] Add per-total-score quartile information graphs for questions~~ Analytics: add per-total-score quartile sparklines for each question Mar 24, 2026

Copilot AI requested a review from inducer March 24, 2026 17:24

Copilot finished work on behalf of inducer March 24, 2026 17:24

inducer force-pushed the copilot/add-quartile-score-graphs branch from 339beee to 84d6de7 Compare March 26, 2026 20:53

Format biome config

07aec35

inducer force-pushed the copilot/add-quartile-score-graphs branch 2 times, most recently from da540b7 to e7cf706 Compare March 26, 2026 21:37

inducer marked this pull request as ready for review March 26, 2026 21:39

Copilot AI review requested due to automatic review settings March 26, 2026 21:39

Copilot started reviewing on behalf of inducer March 26, 2026 21:40 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

Copilot started work on behalf of inducer March 26, 2026 21:59 View session

Copilot finished work on behalf of inducer March 26, 2026 22:06

Copilot AI and others added 2 commits March 26, 2026 17:09

Optimize flow analytics via a subquery

a526032

inducer force-pushed the copilot/add-quartile-score-graphs branch from 3fc40eb to 93cb43d Compare March 26, 2026 22:10

Update baseline

bb5d181

inducer force-pushed the copilot/add-quartile-score-graphs branch from 93cb43d to bb5d181 Compare March 27, 2026 02:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Analytics: add per-total-score quartile sparklines for each question#1449

Analytics: add per-total-score quartile sparklines for each question#1449
Copilot wants to merge 4 commits intomainfrom
copilot/add-quartile-score-graphs

Copilot AI commented Mar 24, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 26, 2026

Uh oh!

inducer Mar 26, 2026

Uh oh!

Copilot AI Mar 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comments on the Issue (you are @copilot in this section)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

inducer Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 24, 2026 •

edited

Loading