Implement a basic HTTP memory cache #4117
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… (pre-redirect) URL and request headers and only stores GET responses.
…g against doomed cache enties.
…dated. Correct inverted expiry check in revalidation code.
…nt failures due to concurrent cache tests.
|
Critic review: https://critic.hoppipolla.co.uk/r/3296 This is an external review system which you may optionally use for the code review of your pull request. In order to help critic track your changes, please do not make in-place history rewrites (e.g. via |
| //TODO: Last-Modified | ||
| //TODO: Range requests | ||
| //TODO: Revalidation rules for query strings | ||
| //TODO: Vary |
|
@jdm First pass of review finished—let me know what spec I should be looking at and I can review the logic. |
|
I've been going off of the old RFC 2616, but it looks like http://tools.ietf.org/html/rfc7234 and http://tools.ietf.org/html/rfc7232 are the new specs that should apply here. |
|
(also based on code inspection of nsHttpChannel.cpp in Gecko) |
|
@jdm OK, the logic looks good per those specs. |
jdm
added
S-needs-code-changes
This was referenced Jan 11, 2015
|
I'll rebase this at some point. Until then, no need to keep this open. |
bors-servo
pushed a commit
that referenced
this pull request
Nov 12, 2017
[WIP] Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
bors-servo
pushed a commit
that referenced
this pull request
Nov 15, 2017
[WIP] Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
bors-servo
pushed a commit
that referenced
this pull request
Nov 15, 2017
[WIP] Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
bors-servo
pushed a commit
that referenced
this pull request
Nov 15, 2017
[WIP] Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
bors-servo
pushed a commit
that referenced
this pull request
Nov 15, 2017
[WIP] Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
bors-servo
pushed a commit
that referenced
this pull request
Nov 19, 2017
Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
bors-servo
pushed a commit
that referenced
this pull request
Nov 21, 2017
Continue http cache work <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of #4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18676) <!-- Reviewable:end -->
moz-v2v-gh
pushed a commit
to mozilla/gecko-dev
that referenced
this pull request
Nov 21, 2017
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7 --HG-- extra : subtree_source : https%3A//hg.mozilla.org/projects/converted-servo-linear extra : subtree_revision : 059fa41ef6c333ab9cc0adece4f431fec59a84c2
xeonchen
pushed a commit
to xeonchen/gecko-cinnabar
that referenced
this pull request
Nov 21, 2017
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7
aethanyc
pushed a commit
to aethanyc/gecko-dev
that referenced
this pull request
Nov 22, 2017
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7
JerryShih
pushed a commit
to JerryShih/gecko-dev
that referenced
this pull request
Nov 28, 2017
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7
gecko-dev-updater
pushed a commit
to marco-c/gecko-dev-wordified-and-comments-removed
that referenced
this pull request
Oct 2, 2019
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7 UltraBlame original commit: 8032f2843df37d05efa56b3e83a38b326789b8b2
gecko-dev-updater
pushed a commit
to marco-c/gecko-dev-comments-removed
that referenced
this pull request
Oct 2, 2019
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7 UltraBlame original commit: 8032f2843df37d05efa56b3e83a38b326789b8b2
gecko-dev-updater
pushed a commit
to marco-c/gecko-dev-wordified
that referenced
this pull request
Oct 2, 2019
…e_http_cache_work); r=jdm <!-- Please describe your changes on the following line: --> Work in progress, and not quite worth a review yet. (Continuation of servo/servo#4117) TODO - [ ] cache metadata (find some subset of`net_traits::Metadata` that can be shared across threads, it seems the problem is mainly stuff inside `hyper::header` in the `headers` field) - [ ] determine which other fields of a `Response` need to be cached, so a full and valid one can be returned upon a cache hit. - [ ] determine how to best share the cache across fetch threads (inside HttpState like I tried now?) - [ ] Spend more time reading the spec and make sure the cache follows it where it matters. - [ ] Make the current wpt tests pass. - [ ] More... --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #12972 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____ <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: e2bc0f017cfd24734777f64d8607b23dbe9552d7 UltraBlame original commit: 8032f2843df37d05efa56b3e83a38b326789b8b2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
High-level design:
This design handles multiple simultaneous requests for an in-progress cached response; each time a new portion of the body is stored, the list of awaiting consumers will be notified. It is also quite amenable to future changes to use asynchronous, push-based network operations.
Additionally, this changeset enables some really simple tests of caching behaviour by making all content tests run under a custom webserver that can track the number of full responses for a particular resource.