Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP cache control #833

Merged
merged 16 commits into from
Oct 20, 2022
Merged

HTTP cache control #833

merged 16 commits into from
Oct 20, 2022

Conversation

veloman-yunkan
Copy link
Collaborator

@veloman-yunkan veloman-yunkan commented Oct 9, 2022

Fixes #650
Fixes #831

@codecov
Copy link

codecov bot commented Oct 9, 2022

Codecov Report

Base: 70.60% // Head: 70.96% // Increases project coverage by +0.35% 🎉

Coverage data is based on head (18a18c1) compared to base (7feef32).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #833      +/-   ##
==========================================
+ Coverage   70.60%   70.96%   +0.35%     
==========================================
  Files          53       53              
  Lines        3664     3699      +35     
  Branches     2046     2063      +17     
==========================================
+ Hits         2587     2625      +38     
+ Misses       1075     1072       -3     
  Partials        2        2              
Impacted Files Coverage Δ
include/library.h 100.00% <ø> (ø)
src/server/internalServer.h 89.47% <ø> (ø)
src/library.cpp 83.52% <100.00%> (+0.11%) ⬆️
src/server/etag.cpp 95.65% <100.00%> (ø)
src/server/etag.h 100.00% <100.00%> (ø)
src/server/internalServer.cpp 88.51% <100.00%> (+0.42%) ⬆️
src/server/internalServer_catalog_v2.cpp 93.10% <100.00%> (+0.08%) ⬆️
src/server/response.cpp 93.95% <100.00%> (+0.29%) ⬆️
src/server/response.h 100.00% <100.00%> (ø)
src/opds_dumper.cpp 99.16% <0.00%> (+0.83%) ⬆️
... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@kelson42
Copy link
Collaborator

kelson42 commented Oct 9, 2022

Fixes #831 and partly fixes #650?! Might be better.

@veloman-yunkan
Copy link
Collaborator Author

Fixes #831 and partly fixes #650?! Might be better.

No. This PR in its current state fixes #831 only partly since it doesn't address the aspects discussed in the questions posed in that ticket. Since the higher level ticket #650 was not concerned with such issues, it can be considered to be fully fixed by this PR.

@veloman-yunkan
Copy link
Collaborator Author

Now #831 is fully fixed by this PR

Copy link
Member

@mgautierfr mgautierfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!! QUESTION: Do we need those resources? !!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

For favicon, I not sure but I think we should keep them.
Browsers may do some kind of auto discovery of the favicon by directly request them even if they are not declared in the html. @kelson42 may have a better idea here.

For search-icon.svg, it is referenced in static/skin/index.css
search.svg seems indeed not used. I think we can remove it.

src/server/internalServer.cpp Outdated Show resolved Hide resolved
@mgautierfr
Copy link
Member

Real favicon generator may help us to know what we need to keep : https://realfavicongenerator.net/favicon_checker?protocol=http&site=dev.library.kiwix.org

Copy link
Member

@mgautierfr mgautierfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have one open question but else, it is a approval.

static/skin/index.css Show resolved Hide resolved
@mgautierfr
Copy link
Member

Please rebase-fixup on new master and we should be good to merge.

Preparing the server unit-test for the more elaborate handling of HTTP
caching.
At this point the ETag value for ZIM content is still generated from the
timestamp of the server start-up time.
One (hopefully, last) remaining relative URL to a static resource
is the reference to ./search-icon.svg found in skin/index.css to which
KIWIXCACHEID could not be applied because of the limitations of the
resource preprocessing script `kiwix-resources`.
During static resource preprocessing and compilation their cacheid
values are embedded into libkiwix and can be accessed at runtime.

If a static resource is requsted without specifying any cacheid
it is served as dynamic content (with short TTL and the library id
used for the ETag, though using the cacheid for the ETag would
be better).

If a cacheid is supplied in the request it must match the cacheid of the
resource (otherwise a 404 Not Found error is returned) whereupon the
resource is served as immutable content.

Known issues:

- One issue is caused by the fact that some static resources don't get a
  cacheid; this is resolved in the next commit.

- Interaction of this change with the support for dynamically customizing
  static resources (via KIWIX_SERVE_CUSTOMIZED_RESOURCES env var) was
  not addressed.
Before this change cacheids were computed only for those static
resources that were referenced from other resources via KIWIXCACHEID.

A few static resources without such references existed.

Now all resources under skin/ have their cacheids computed.
@veloman-yunkan
Copy link
Collaborator Author

Please rebase-fixup on new master and we should be good to merge.

Done. The only change in addition to rebase-fixup is the edited commit message of 415ec41 (the question answered by #833 (review) was removed).

@mgautierfr mgautierfr merged commit 0e20f50 into master Oct 20, 2022
@mgautierfr mgautierfr deleted the http_caching branch October 20, 2022 14:17
mgautierfr added a commit that referenced this pull request Nov 30, 2022
* [API Break] Remove wrapper around libzim (@mgautierfr #789)
* Allow kiwix-serve to use custom resource files (@veloman-yunkan #779)
* Properly handle searchProtocolPrefix when rendering search result (@veloman-yunkan #823)
* Prevent search on multi language content (@veloman-yunkan #838)
* Use new `zim::Archive::getMediaCount` from libzim (@mgautierfr #836)
* Catalog:
 - Include tags in free text catalog search (@veloman-yunkan #802)
 - Illustration's url is based on book's uuid (@veloman-yunkan #804)
 - Cleanup of the opds-dumper (@veloman-yunkan #829)
 - Allow filtering of catalog content using multiple languages (@veloman-yunkan #841)
 - Make opds-dumper respect the namemapper (@mgautierfr #837)
* Server:
 - Correctly handle `\` in suggestion json generation (@veloman-yunkan #843)
 - Better http caching (@veloman-yunkan #833)
 - Make `/suggest` endpoint thread-safe (@veloman-yunkan #834)
 - Better redirection of main page (@veloman-yunkan #827)
 - Remove jquery (@mgautierfr @juuz0 #796)
 - Better Viewer of zim content :
   . Introduce `/content` endpoints (@veloman-yunkan #806)
   . Switch to iframe based content viewer (@veloman-yunkan #716)
 - Optimised design of the welcome page:
   . Alignement (@juuz0 @kelson42 #786)
   . Exit download modal on pressing escape key (@Juzz0 #800)
   . Add favicon for different devices (@Juzz0 #805)
   . Fix auto hidding of the toolbar (@veloman-yunkan #821)
   . Allow user to filter books by tags in the front page (@juuz0 #711)
* CI :
  - Trigger CI on pull_request (@kelson42 #791)
  - Drop Ubuntu Impish packaging (@legoktm #825)
  - Add Ubuntu Kinetic packaging (@legoktm #801)
* Testing:
  - Test ICULanguageInfo (@veloman-yunkan #795)
  - Introduce fake `test` language to test i18n (@veloman-yunkan #848)
* Fix documentation (@kelson42 #816)
* Udpate translation (#787 #839 #847)
@mgautierfr mgautierfr mentioned this pull request Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cache control of static resources Update Kiwix-serve cache strategy
3 participants