Reduce N+1s by caching submitted project count (Part 1 of 2)#804
Open
zetter-rpf wants to merge 2 commits intomainfrom
Open
Reduce N+1s by caching submitted project count (Part 1 of 2)#804zetter-rpf wants to merge 2 commits intomainfrom
zetter-rpf wants to merge 2 commits intomainfrom
Conversation
zetter-rpf
changed the title
Test coverage89.46% line coverage reported by SimpleCov. |
Contributor
There was a problem hiding this comment.
Pull request overview
Introduces a cached submitted_projects_count on Lesson and keeps it updated when SchoolProject transitions into/out of submitted, plus a rake task + specs to backfill/verify counts to support eliminating N+1 queries in controllers.
Changes:
- Add
lessons.submitted_projects_countcolumn (default 0) and aLesson#recalculate_submitted_projects_count!helper. - Trigger lesson count recalculation on
SchoolProjectstate transitions to/fromsubmitted. - Add
lessons:backfill_submitted_projects_countrake task and RSpec coverage for the model/state machine/task.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| app/models/lesson.rb | Adds remix/school_project associations and count recalculation method for the new cached column. |
| app/models/school_project.rb | Adds lesson lookup and a callback helper used by the state machine. |
| app/state_machines/school_project_state_machine.rb | Hooks after_transition callbacks for submitted-enter/leave events. |
| lib/tasks/lessons.rake | Adds backfill task to populate cached counts in bulk via SQL. |
| db/migrate/20260429120000_add_submitted_projects_count_to_lessons.rb | Adds the new cached count column. |
| db/schema.rb | Reflects the new lessons column and schema version bump. |
| spec/models/lesson_spec.rb | Adds specs for #recalculate_submitted_projects_count!. |
| spec/state_machines/school_project_state_machine_spec.rb | Adds specs ensuring transitions update the cached lesson count. |
| spec/lib/tasks/lessons_spec.rb | Adds rake task spec for the backfill behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There's currently N+1 queries on both the lessons and classes controller as they both try to show the number of submitted projects. Originally I tired to solve this by using 'include' to load the correct records, however I don't think this is a very scalable solutions as it means loading every submitted project object memory for classes or lessons (which there may be 1000s). Instead use the pattern of a counter cache. Each time a project transitions to/from submitted, we recalculate the number. This is the first part of two. I've broken up the work so we can deploy this and check that the counts are correct. If we deployed this in one go there would be a chance the counts are inaccurate if a project transition happened during the deploy. ### After deploy Run `rails lessons:backfill_submitted_projects_count` and verify the results by checking a few lessons and comparing it to `lesson#submitted_count` Co-authored-by: Copilot <copilot@github.com>
Add lock to prevent stale reads in the case this runs concurrently for the same project Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
zetter-rpf
force-pushed
the
cache-submitted-count
branch
from
9def7db to
6ff71db
Compare
zetter-rpf
added a commit
that referenced
this pull request
May 1, 2026
In [1] we started caching the value of submitted projects. Now we can use that value to simplify the code and reduce N+1 queries. [1] #804 Note that I've had to split the loading of remixes for teachers and students as teachers no longer need remixes (which causes bullet to complain) Co-authored-by: Copilot <copilot@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status
What's changed?
There's currently N+1 queries on both the lessons and classes controller as they both try to show the number of submitted projects.
Originally I tired to solve this by using 'include' to load the correct records, however I don't think this is a very scalable solutions as it means loading every submitted project object memory for classes or lessons (which there may be 1000s, especially when listing classes)
Instead use the pattern of a counter cache. Each time a project transitions to/from submitted, we recalculate the number.
This is the first part of two. I've broken up the work so we can deploy this and check that the counts are correct. If we deployed this in one go there would be a chance the counts are inaccurate if a project transition happened during the deploy.
After deploy
Run
rails lessons:backfill_submitted_projects_countand verify the results by checking a few lessons and comparing it tolesson#submitted_count