Consider using xz to compress production assets

[abarth]

As an example, xz seems to compress the ICU data table better than zip:

2.1M Dec 11 20:17 icudtl.dat.zip
1.4M Dec 11 20:18 icudtl.dat.xz

We could use xz to compress the data table and then place it in the APK without further compression.

See http://tukaani.org/xz/embedded.html

/cc @eseidel

[abarth]

We also might want to use it for snapshots. Here's how it compresses the Stocks snapshot:

332K Dec 11 20:57 snapshot_blob.bin.xz
419K Dec 11 20:57 snapshot_blob.bin.zip

[sethladd]

Curious, did we look at https://github.com/google/zopfli ?

Also, https://github.com/google/brotli/ ?

[abarth]

Can evaluate zopfli, but it has the added restriction that it uses zlib for decompression. I suspect xz will give a better compression ratio because it's allowed to change the decompression library.

brotli isn't good for our purposes because the decoder is huge (>100KB). The decoder has a giant initial dictionary: https://github.com/google/brotli/blob/master/dec/dictionary.c

[eseidelGoogle]

@chinmaygarde also told me today that he may be able to cut our ICU data table by as much as half by turning off some ICU features we aren't using.

[eseidelGoogle]

Will this go away if @szakarias is successful at removing FLXs?

[chinmaygarde]

We don't package the ICU data tables in the FLX. Instead, it is present in the application package (APK or IPA). Also, the script snapshots are no longer present in the FLX files in product and release modes. We should close this issue.

[eseidelGoogle]

@jason-simmons has considered changing our assets code to read ICU data directly from the .apk without unpacking. I think we do that with other assets already?