Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Re-optimize alphabet for Brotli (only chars unused by gzip backrefs) #310

Merged
merged 2 commits into from
Oct 13, 2021

Conversation

subzey
Copy link
Contributor

@subzey subzey commented Oct 13, 2021

A follow-up for #309: Although the Brotli size is currently not counted, probably this would be nice to have.

The "brotlied" built non-secure nanoid size goes down from 146 bytes to 129.

The gzip sizes are unchanged for all entries.

How does it work

Currently only 'use, andom and rict' are used to utilize the gzip backrefs. Other chars are just sorted alphabetically (they cannot form a gzip backref so their order is not significant for gzip).

But we also have brotli! And brotli has a built-in the default dictionary. All we need is to rerrange those "other chars" to form some substrings from the dictionary.

After a day running an ad-hoc script gave the answer for the Ultimate Question of Life, the Universe, and Everything this bizarre Scrabble game:

-26T
1983
40px (uppercased)
75px
bush (uppercased)
jack (uppercased)
mind (uppercased)
very (uppercased)
wolf (uppercased)

That forms a "phrase" -26T198340PX75pxJACKVERYMINDBUSHWOLF

Equally good, also producing 129 bytes
-26T139475px80PXBINDGULFJACKVERSWHOM
-26T139475px80PXFINDJOBSMUCHVERYWALK
-26T139475PX80pxBINDGULFJACKVERSWHOM
-26T139475PX80pxFINDJOBSMUCHVERYWALK
-26T198340px75PXBINDGULFJACKVERSWHOM
-26T198340PX75pxBINDGULFJACKVERSWHOM
-26T198340PX75pxBUSHDOWNFILMJACKVERY
-26T198340PX75pxBUSHFILMJACKNDOWVERY
-26T198340PX75pxBUSHFLOWJACKMINDVERY
-26T198430px75PXBINDGULFJACKVERSWHOM
-26T198430PX75pxBINDGULFJACKVERSWHOM
Honorable mention (132 bytes)
198640PX75pxBIRDFUCKGAVEHTMLJSON

@ai
Copy link
Owner

ai commented Oct 13, 2021

Can you update size-limit config to make PR diff?

@ai
Copy link
Owner

ai commented Oct 13, 2021

And can I ask you to add a note to the code about the reason behind this optimizaion?

@subzey
Copy link
Contributor Author

subzey commented Oct 13, 2021

The size-limit section is unchanged as the GZIPped sizes are the same.

This optimization only affects Brotli size that is currently not measured by size-limit (can it?)

@ai
Copy link
Owner

ai commented Oct 13, 2021

The size-limit section is unchanged as the GZIPped sizes are the same.

Oops. In this case we need only explanation comments.

This optimization only affects Brotli size that is currently not measured by size-limit (can it?)

It can with brotli config option. We can add extra Size Limit field for it.

@subzey subzey force-pushed the feat/alphabet-brotli branch from 1e5415d to 1fd06c2 Compare October 13, 2021 10:46
@subzey
Copy link
Contributor Author

subzey commented Oct 13, 2021

I've force-pushed two commits:

8140bc8 Start measuring brotli in size-limit
1fd06c2 The implementation and size-limit section diff

@ai ai merged commit 75237a6 into ai:main Oct 13, 2021
@ai
Copy link
Owner

ai commented Oct 13, 2021

I combined brotli metrics since it is non-main use case (yeap, it will not be too accurate combined)

@ai
Copy link
Owner

ai commented Oct 13, 2021

Thanks. Released in 3.1.30.

@subzey
Copy link
Contributor Author

subzey commented Oct 17, 2021

Agree, this config looks more than enough for a good-to-have feature. Thanks!

@subzey subzey deleted the feat/alphabet-brotli branch October 17, 2021 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants