Skip to content

Add brotli transform to compress the files using brotli algorithm#62

Merged
jkent merged 1 commit intojkent:mainfrom
X-Ryl669:addBrotliTransform
Aug 19, 2024
Merged

Add brotli transform to compress the files using brotli algorithm#62
jkent merged 1 commit intojkent:mainfrom
X-Ryl669:addBrotliTransform

Conversation

@X-Ryl669
Copy link
Copy Markdown
Contributor

Purpose

With new code, frogfs is much better tooled for embedded storage of a website.
On the web, the best supported compression algorithm is brotli. All browsers support it (96.93% support in caniuse), and it provides a strong compression advantage (for example, gzip encoded JS file will take 660 bytes while brotli will only use 440 bytes)

So this PR mainly add brotli as a transformation filter, like gzip and deflate used to do it.

It doesn't add any code in the microcontroller for decompressing the brotli's data, it's only used as a preprocessing step to reduce the filesystem size. The microcontroller will simply send the data as-is when requested.

Fixes

While testing this PR, I've found few subtle bugs that I corrected.

Transform order wasn't respected

Let's say you had this in your YAML file:

[...]
filters:
  '*': 
    - gzip
  
  '*.css':
    - uglifycss
    - gzip

In the current code, the later filter on CSS files will apply uglifycss on a gzipped input. That because, when the xforms dict is created in apply_rules, it first find the '*' filter, thus a gzip is transform is created. Then it finds the '*.css' filter, and create the uglifycss transform (it later find the gzip filter again, but since it already exists, it only replace it, not changing the order in the dictionary). You end up with a dict containing gzip, uglifycss in that order.

I've solved that simply by ranking the filters. A filter with a higher rank will overwrite the xforms dict thus preventing bad transforms order in the dictionary. A lower rank filter will be ignored if an higher rank filter was processed before it. I think it's the most logical solution, since you don't want to mix the actions of filters with higher specializations.

no transform crashed the processing

In the current code, if you have a no something filter, the ent["transform"] will contain a False argument (as a way to prevent processing this action). Yet, the actual code doing the processing was only expecting a dict, crashing the processing. Now, the processing handles the case carefully by bypassing the processing as expected.

Fixing spell error

…ch better than gzip and supported everywhere)
Copy link
Copy Markdown
Owner

@jkent jkent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you once again X-Ryl669!

@jkent jkent merged commit c2f1419 into jkent:main Aug 19, 2024
@jkent jkent moved this to To triage in ACEWS Ecosystem Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants