Support for serving customized resources #779
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #779 +/- ##
==========================================
+ Coverage 63.66% 64.21% +0.54%
==========================================
Files 59 59
Lines 4073 4102 +29
Branches 2218 2227 +9
==========================================
+ Hits 2593 2634 +41
+ Misses 1478 1466 -12
Partials 2 2
Continue to review full report at Codecov.
|
veloman-yunkan
force-pushed
the
serving_customized_resources
branch
from
1a7808e
to
7b7e4e2
Compare
|
This PR originates from #716 (comment). The new functionality in its current form doesn't fully take over the old solution that's why I didn't remove the old solution from |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions. |
|
@mgautierfr Can you please review this Pr? |
kelson42
force-pushed
the
serving_customized_resources
branch
from
7b7e4e2
to
a6a2704
Compare
During work on the kiwix-serve front-end, the edit-save-test cycle is a multistep procedure: 1. build and install libkiwix 2. build kiwix-tools 3. run kiwix-serve 4. reload the web-page in the browser When making changes in static resources that are served by kiwix-serve unmodified, the steps 1-3 can be eliminated if kiwix-serve is capable of serving resources from the file-system. This commit adds such a functionality to kiwix-serve. Now, if during startup of kiwix-serve the environment variable `KIWIX_SERVE_CUSTOMIZED_RESOURCES` is defined it is assumed to point to a file where every line has the following format: URL MIMETYPE RESOURCE_FILE_PATH When a request is received by kiwix-serve and its URL matches any of the URLs read from the customized resource file, then the resource data is read from the respective file RESOURCE_FILE_PATH and served with mime-type MIMETYPE. Though this feature was introduced in order to facilitate the development of the iframe-based content viewer, it can also be useful to users who would like to customize the kiwix-serve front-end on their own (without re-building all of kiwix-serve). There is some overlap with a feature of the kiwix-compile-resources script that also allows to override resources. The differences are: 1. The new way of customizing front-end resources has all such resources listed in a text file and there is a single environment variable from which the path of that file is read. kiwix-compile-resources associates a separate environment variable with each resource. 2. The new way uses regular paths to identify a resource. The kiwix-compile-resources method encodes the resource path by replacing any non-alphanumeric characters (including the path separator) with underscores (so that the resulting resource identifier can be used to construct the name of the environment variable controlling that resource). 3. The new method allows adding new front-end resources. The old method only allows to modify existing resources. 4. The new method allows (actually requires) to specify the URL at which the overriden resource should be served (similarly, the MIME-type can/must be specified, too). The old method only allows to override the contents of a resource. 5. The new method only allows to override front-end resources that are served without any preprocessing by kiwix-serve at runtime. The old method allows to override template resources as well (note that internationalization/translation resources cannot be overriden using the old method, either).
kelson42
force-pushed
the
serving_customized_resources
branch
from
a6a2704
to
e3e4bfa
Compare
mgautierfr
previously approved these changes
Jun 22, 2022
One important missing test is that the content of the customized resource is read from storage every time rather than once. Testing that requirement would involve creating temporary files which is a little more work.
Added a couple of tests that provide only basic coverage of the new code. The last commit outlines an aspect of the desired functionality that is not tested. |
mgautierfr
approved these changes
Jun 22, 2022
|
Agree with the message of the last commit. Testing that file is read each request would be a bit too complex to test. |
mgautierfr
added a commit
that referenced
this pull request
Nov 30, 2022
* [API Break] Remove wrapper around libzim (@mgautierfr #789) * Allow kiwix-serve to use custom resource files (@veloman-yunkan #779) * Properly handle searchProtocolPrefix when rendering search result (@veloman-yunkan #823) * Prevent search on multi language content (@veloman-yunkan #838) * Use new `zim::Archive::getMediaCount` from libzim (@mgautierfr #836) * Catalog: - Include tags in free text catalog search (@veloman-yunkan #802) - Illustration's url is based on book's uuid (@veloman-yunkan #804) - Cleanup of the opds-dumper (@veloman-yunkan #829) - Allow filtering of catalog content using multiple languages (@veloman-yunkan #841) - Make opds-dumper respect the namemapper (@mgautierfr #837) * Server: - Correctly handle `\` in suggestion json generation (@veloman-yunkan #843) - Better http caching (@veloman-yunkan #833) - Make `/suggest` endpoint thread-safe (@veloman-yunkan #834) - Better redirection of main page (@veloman-yunkan #827) - Remove jquery (@mgautierfr @juuz0 #796) - Better Viewer of zim content : . Introduce `/content` endpoints (@veloman-yunkan #806) . Switch to iframe based content viewer (@veloman-yunkan #716) - Optimised design of the welcome page: . Alignement (@juuz0 @kelson42 #786) . Exit download modal on pressing escape key (@Juzz0 #800) . Add favicon for different devices (@Juzz0 #805) . Fix auto hidding of the toolbar (@veloman-yunkan #821) . Allow user to filter books by tags in the front page (@juuz0 #711) * CI : - Trigger CI on pull_request (@kelson42 #791) - Drop Ubuntu Impish packaging (@legoktm #825) - Add Ubuntu Kinetic packaging (@legoktm #801) * Testing: - Test ICULanguageInfo (@veloman-yunkan #795) - Introduce fake `test` language to test i18n (@veloman-yunkan #848) * Fix documentation (@kelson42 #816) * Udpate translation (#787 #839 #847)
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
During work on the kiwix-serve front-end, the edit-save-test cycle is a multistep procedure:
When making changes in static resources that are served by kiwix-serve unmodified, the steps 1-3 can be eliminated if kiwix-serve is capable of serving resources from the file-system. This commit adds such a functionality to kiwix-serve. Now, if during startup of kiwix-serve the environment variable
KIWIX_SERVE_CUSTOMIZED_RESOURCESis defined it is assumed to point to a file here every line has the following format:URL MIMETYPE RESOURCE_FILE_PATH
When a request is received by kiwix-serve and its URL matches any of the URLs read from the customized resource file, then the resource data is read from the respective file RESOURCE_FILE_PATH and served with mime-type MIMETYPE.
Though this feature was introduced in order to facilitate the development of the iframe-based content viewer, it can also be useful to users who would like to customize the kiwix-serve front-end on their own (without re-building all of kiwix-serve).
There is some overlap with a feature of the kiwix-compile-resources script that also allows to override resources. The differences are:
The new way of customizing front-end resources has all such resources listed in a text file and there is a single environment variable from which the path of that file is read. kiwix-compile-resources associates a separate environment variable with each resource.
The new way uses regular paths to identify a resource. The kiwix-compile-resources method encodes the resource path by replacing any non-alphanumeric characters (including the path separator) with underscores (so that the resulting resource identifier can be used to construct the name of the environment variable controlling that resource).
The new method allows adding new front-end resources. The old method only allows to modify existing resources.
The new method allows (actually requires) to specify the URL at which the overriden resource should be served (similarly, the MIME-type can/must be specified, too). The old method only allows to override the contents of a resource.
The new method only allows to override front-end resources that are served without any preprocessing by kiwix-serve at runtime. The old method allows to override template resources as well (note that internationalization/translation resources cannot be overriden using the old method, either).