Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 12 hour caching to our robots.txt serving #6876

Merged
merged 4 commits into from Apr 13, 2020
Merged

Conversation

ericholscher
Copy link
Member

@ericholscher ericholscher commented Apr 7, 2020

This should help reduce load on our servers when serving robots.txt.
This file should change very slowly,
and when it does it's not super important for it to change quickly.

This also makes sitemaps the same 12 hours,
since 3 days seemed a bit aggressive.

This should help reduce load on our servers when serving robots.txt.
This file should change very slowly,
and when it does it's not super important for it to change quickly.
@ericholscher
Copy link
Member Author

@ericholscher ericholscher commented Apr 7, 2020

This needs the tests to be updated. I'm guessing they are now being cached :) Not sure the best approach -- perhaps using a null cache for the tests?

Copy link
Member

@humitos humitos left a comment

Looks good!

Not sure the best approach -- perhaps using a null cache for the tests?

Maybe using @override_settings with CACHES={} on these tests?

@@ -355,7 +356,7 @@ class ServeRobotsTXT(SettingsOverrideObject):
class ServeSitemapXMLBase(View):

@method_decorator(map_project_slug)
@method_decorator(cache_page(60 * 60 * 24 * 3)) # 3 days
@method_decorator(cache_page(60 * 60 * 12)) # 12 hours
Copy link
Member

@humitos humitos Apr 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense. We are communicating in the sitemap for latest to "daily".

@ericholscher
Copy link
Member Author

@ericholscher ericholscher commented Apr 13, 2020

Maybe using @override_settings with CACHES={} on these tests?

I tried that and it didn't seem to work, I am explicitly clearing the cache now which worked locally 👍

@ericholscher ericholscher merged commit d5d753d into master Apr 13, 2020
2 checks passed
@ericholscher ericholscher deleted the cache-robotstxt branch Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants