Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce robots.txt cache TTL #7334

Merged
merged 1 commit into from Jul 29, 2020
Merged

Conversation

agjohnson
Copy link
Contributor

@agjohnson agjohnson commented Jul 29, 2020

The TTL on the cache_page call of 12 hours exceeds the expiration of the
S3 presign URL expiration TTL. I believe the default here is the
AWS_QUERYSTRING_EXPIRE setting, default 3600s:
https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings

Going beyond expiration on the presign URL results in a broken response
to the user, so not something we can safely keep cached.

12 hours is likely far too agressive here anyways. We could likely drop
this to a sub-5m TTL. If this is cached because it's an expensive view,
an hour seems like a good compromise for now.

The TTL on the cache_page call of 12 hours exceeds the expiration of the
S3 presign URL expiration TTL. I believe the default here is the
`AWS_QUERYSTRING_EXPIRE` setting, default 3600s:
https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html#settings

Going beyond expiration on the presign URL results in a broken response
to the user, so not something we can safely keep cached.

12 hours is likely far too agressive here anyways. We could likely drop
this to a sub-5m TTL. If this is cached because it's an expensive view,
an hour seems like a good compromise for now.
@agjohnson agjohnson requested a review from humitos Jul 29, 2020
@ericholscher ericholscher merged commit 63edf91 into master Jul 29, 2020
2 checks passed
@ericholscher ericholscher deleted the agj/reduce-robots-cache-ttl branch Jul 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants