Skip to content

Clear Cache on Deploy #1668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 10, 2024
Merged

Clear Cache on Deploy #1668

merged 3 commits into from
Oct 10, 2024

Conversation

@shanbady
Copy link
Contributor

@shanbady shanbady commented Oct 9, 2024

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/5743

Description (What does it do?)

This pr adds a django command that is run on each deploy to clear the cache.

The site-wide page cache gets invalidated when etl pipelines run and when search indexing operations complete however - we recently ran into an issue where an api endpoint added a new attribute and due to caching, the api responded with a cached response and a frontend feature that relied on this broke.

How can this be tested?

To test locally:

  1. checkout this branch
  2. browse around some pages in order to have some items in the cache
  3. run the cache clear management command python manage.py clear_cache - you should see some output indicating the number of items cleared.

@shanbady shanbady added the Needs Review An open Pull Request that is ready for review label Oct 9, 2024
@shanbady shanbady marked this pull request as ready for review October 9, 2024 14:00
gumaerc
gumaerc approved these changes Oct 10, 2024
View reviewed changes
Copy link
Contributor

@gumaerc gumaerc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 LGTM

The only thing I might suggest here; if you expect this cache clearing function to take a long time in the event that there are a lot of objects to clear, it might be worth putting this into a celery task. Otherwise, it can stay as-is. We'll have to see how this works on RC to test the Heroku component.

@shanbady
Copy link
Contributor Author

shanbady commented Oct 10, 2024

👍 LGTM

The only thing I might suggest here; if you expect this cache clearing function to take a long time in the event that there are a lot of objects to clear, it might be worth putting this into a celery task. Otherwise, it can stay as-is. We'll have to see how this works on RC to test the Heroku component.

I was thinking something like that as well if it takes long. Either celery task (easier) or if we are only going to be dealing with redis manually calling unlink instead of delete

@shanbady shanbady merged commit 793644e into main Oct 10, 2024
11 checks passed
@shanbady shanbady deleted the shanbady/clear-cache-on-deploy branch October 10, 2024 16:00
This was referenced Oct 10, 2024
Closed
Merged
jonkafton added a commit that referenced this pull request Oct 15, 2024
@jonkafton @mbertrand
@odlbot @gumaerc @abeglova @rhysyngsun @shanbady @pre-commit-ci @renovate @jkachel
Merge main into nextjs (#1687)
f0211ef 
* Add location field to LearningResource and LearningResourceRun models (#1604)

* Always delete past events during ETL, and filter them out from api results too just in case (#1660)

* Release 0.21.1

* Release date for 0.21.1

* remove unnecessary padding adjustment on Input base styles (#1666)

* remove unnecessary padding adjustment on Input base styles

* adjust actual textarea padding to match Figma

* Content File Score Adjustment (#1667)

* Release 0.21.2

* Restore program letter intercept view (#1643)

* Restore program letter intercept view

* Generate schema

* Fix test

* fixing import

* Revert "fixing import"

This reverts commit cf56807.

---------

Co-authored-by: shankar ambady <ambady@mit.edu>

* Release date for 0.21.2

* topic detail banner / subtopic logic revisions (#1646)

* add new darker chip variant

* move subtopic display to the banner

* remove max width on banner subtitle texts

* query subtopics only on parent topic channels, and on child topic channels query "related topics" aka the topic's siblings

* don't display the current topic in related topics

* show parent topic breadcrumbs on child topic channel pages

* fix tests

* only do subtopics mocking if necessary

* fix editchannelpage test

* move action buttons under the title on mobile

* fix test

* add new default channel background image

* "toopics"

* fix background position

* remove accidentally committed file

* fix background size on mobile

* remove unnecessary order css rule

* fix banner background on mobile

* fix breadcrumb props

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unnecessary topic id filter

* remove unnecessary breadcrumb fallback

* set the default banner background image in the banner component itself, and only apply large and up 140% background size if the default image is being used

* properly set default background image

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update Yarn to v4.5.0 (#1624)

* Update Yarn to v4.5.0

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Clear Cache on Deploy (#1668)

* adding initial cache clear command

* adding cache clear command to heroku release script

* fix docstrings

* removing invalid flag from clear cache command (#1675)

* prevent featured course carousel from re-randomizing (#1673)

* use skipFeatured when invalidating userlists / learning paths after update

* add a new function for updating query data for user_list_parents and learning_path_parents on the featured courses list when either is edited without re-randomizing

* fix learning path invalidations

* fix test

* Shuffling around where the search_update event is fired so it happens in more places (#1679)

- Changed HeroSearch to use the SearchField component instead of SearchInput, so we get the event capture stuff
- Updated SearchInput to not require setPage (for use on the homepage)
- Moved facet events out of the SearchPage and into SearchDisplay so they get called on other pages that use it
- Also trigger the facet events on more things - should also now get sorting changes, choosing between categories, and clearing facets

* Reinstate background steps image

---------

Co-authored-by: Matt Bertrand <mrbertrand@gmail.com>
Co-authored-by: Doof <mitx-devops@mit.edu>
Co-authored-by: Carey P Gumaer <gumaerc@mit.edu>
Co-authored-by: Anastasia Beglova <abeglova@mit.edu>
Co-authored-by: Nathan Levesque <rhysyngsun@users.noreply.github.com>
Co-authored-by: shankar ambady <ambady@mit.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Shankar Ambady <shanbady@gmail.com>
Co-authored-by: James Kachel <james@jkachel.com>
jonkafton added a commit that referenced this pull request Oct 21, 2024
@jonkafton @mbertrand
@odlbot @gumaerc @abeglova @rhysyngsun @shanbady @pre-commit-ci @renovate @jkachel @ChristopherChudzicki
Merge pull request #1706 from mitodl/jk/5738-ci-config-nextjs-prod
6965103 
* Merge `main` into `nextjs` (#1687)

* Add location field to LearningResource and LearningResourceRun models (#1604)

* Always delete past events during ETL, and filter them out from api results too just in case (#1660)

* Release 0.21.1

* Release date for 0.21.1

* remove unnecessary padding adjustment on Input base styles (#1666)

* remove unnecessary padding adjustment on Input base styles

* adjust actual textarea padding to match Figma

* Content File Score Adjustment (#1667)

* Release 0.21.2

* Restore program letter intercept view (#1643)

* Restore program letter intercept view

* Generate schema

* Fix test

* fixing import

* Revert "fixing import"

This reverts commit cf56807.

---------

Co-authored-by: shankar ambady <ambady@mit.edu>

* Release date for 0.21.2

* topic detail banner / subtopic logic revisions (#1646)

* add new darker chip variant

* move subtopic display to the banner

* remove max width on banner subtitle texts

* query subtopics only on parent topic channels, and on child topic channels query "related topics" aka the topic's siblings

* don't display the current topic in related topics

* show parent topic breadcrumbs on child topic channel pages

* fix tests

* only do subtopics mocking if necessary

* fix editchannelpage test

* move action buttons under the title on mobile

* fix test

* add new default channel background image

* "toopics"

* fix background position

* remove accidentally committed file

* fix background size on mobile

* remove unnecessary order css rule

* fix banner background on mobile

* fix breadcrumb props

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* remove unnecessary topic id filter

* remove unnecessary breadcrumb fallback

* set the default banner background image in the banner component itself, and only apply large and up 140% background size if the default image is being used

* properly set default background image

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update Yarn to v4.5.0 (#1624)

* Update Yarn to v4.5.0

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Clear Cache on Deploy (#1668)

* adding initial cache clear command

* adding cache clear command to heroku release script

* fix docstrings

* removing invalid flag from clear cache command (#1675)

* prevent featured course carousel from re-randomizing (#1673)

* use skipFeatured when invalidating userlists / learning paths after update

* add a new function for updating query data for user_list_parents and learning_path_parents on the featured courses list when either is edited without re-randomizing

* fix learning path invalidations

* fix test

* Shuffling around where the search_update event is fired so it happens in more places (#1679)

- Changed HeroSearch to use the SearchField component instead of SearchInput, so we get the event capture stuff
- Updated SearchInput to not require setPage (for use on the homepage)
- Moved facet events out of the SearchPage and into SearchDisplay so they get called on other pages that use it
- Also trigger the facet events on more things - should also now get sorting changes, choosing between categories, and clearing facets

* Reinstate background steps image

---------

Co-authored-by: Matt Bertrand <mrbertrand@gmail.com>
Co-authored-by: Doof <mitx-devops@mit.edu>
Co-authored-by: Carey P Gumaer <gumaerc@mit.edu>
Co-authored-by: Anastasia Beglova <abeglova@mit.edu>
Co-authored-by: Nathan Levesque <rhysyngsun@users.noreply.github.com>
Co-authored-by: shankar ambady <ambady@mit.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Shankar Ambady <shanbady@gmail.com>
Co-authored-by: James Kachel <james@jkachel.com>

* restoring up and down chevrons (#1690)

* Config to set cache control headers (#1700)

* Middleware to set cache headers

* Set headers in the config

* Upgrade Next.js

* Regex to exclude paths with a file extension to apply only to routes

* Remove middleware file

* Instruct to only store in shared cache (not browsers)

* Copy yarn releases to the Docker container (fixes build) (#1703)

* Copy yarn releases

* Build the docker image on all push

* NextJS - re-enable program letter tests (#1696)

* fixing program letter template

* re-enabling test

* adding space

* remove unnecessary webpack customizations (#1704)

* Increase cache duration (#1705)

* Increase cache durations

* Update comment

* Prod deployment. Add Posthog vars

* NextJS Sentry Integration (#1701)

* configure sentry, fix error pages

* add env vars to CI

* add global styles to global-error

* TEMPORARY self destruct button

* cleanup

* cleanup error boundary, other misc cleanup

* remove a few useless things

* add a simple test i guess

* remove intentional error triggers

* Appzi env vars and Sentry config

* Move deploy jobs to respective workflow. Docker build for dry run

---------

Co-authored-by: Matt Bertrand <mrbertrand@gmail.com>
Co-authored-by: Doof <mitx-devops@mit.edu>
Co-authored-by: Carey P Gumaer <gumaerc@mit.edu>
Co-authored-by: Anastasia Beglova <abeglova@mit.edu>
Co-authored-by: Nathan Levesque <rhysyngsun@users.noreply.github.com>
Co-authored-by: shankar ambady <ambady@mit.edu>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Shankar Ambady <shanbady@gmail.com>
Co-authored-by: James Kachel <james@jkachel.com>
Co-authored-by: Chris Chudzicki <christopher.chudzicki@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gumaerc gumaerc gumaerc approved these changes

Assignees

No one assigned

Labels

Needs Review An open Pull Request that is ready for review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shanbady @gumaerc