Add brotli transform to compress the files using brotli algorithm#62
Merged
jkent merged 1 commit intojkent:mainfrom Aug 19, 2024
Merged
Add brotli transform to compress the files using brotli algorithm#62jkent merged 1 commit intojkent:mainfrom
jkent merged 1 commit intojkent:mainfrom
Conversation
…ch better than gzip and supported everywhere)
jkent
approved these changes
Aug 19, 2024
Owner
jkent
left a comment
There was a problem hiding this comment.
Looks good! Thank you once again X-Ryl669!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
With new code, frogfs is much better tooled for embedded storage of a website.
On the web, the best supported compression algorithm is brotli. All browsers support it (96.93% support in caniuse), and it provides a strong compression advantage (for example, gzip encoded JS file will take 660 bytes while brotli will only use 440 bytes)
So this PR mainly add brotli as a transformation filter, like gzip and deflate used to do it.
It doesn't add any code in the microcontroller for decompressing the brotli's data, it's only used as a preprocessing step to reduce the filesystem size. The microcontroller will simply send the data as-is when requested.
Fixes
While testing this PR, I've found few subtle bugs that I corrected.
Transform order wasn't respected
Let's say you had this in your YAML file:
In the current code, the later filter on CSS files will apply uglifycss on a gzipped input. That because, when the
xformsdict is created inapply_rules, it first find the'*'filter, thus agzipis transform is created. Then it finds the'*.css'filter, and create theuglifycsstransform (it later find thegzipfilter again, but since it already exists, it only replace it, not changing the order in the dictionary). You end up with a dict containinggzip,uglifycssin that order.I've solved that simply by ranking the filters. A filter with a higher rank will overwrite the
xformsdict thus preventing bad transforms order in the dictionary. A lower rank filter will be ignored if an higher rank filter was processed before it. I think it's the most logical solution, since you don't want to mix the actions of filters with higher specializations.no transform crashed the processing
In the current code, if you have a
no somethingfilter, theent["transform"]will contain aFalseargument (as a way to prevent processing this action). Yet, the actual code doing the processing was only expecting adict, crashing the processing. Now, the processing handles the case carefully by bypassing the processing as expected.Fixing spell error