Compression Support

[laktak]

I've added compression support (for zip & zlib) using https://github.com/imaya/zlib.js.

This also fixes decompression for large data (images) that jxg.js couldn't handle (and a compData vs compdata typo).

I'm not sure how you want to handle the dependency so I've included the source like you did with jxg.

[toberndo]

Great. This fixes an old compatibility issue.

I'm not sure how you want to handle the dependency so I've included the source like you did with jxg.

Currently the sources of dependencies are included but I think we want to move away from that and use npm dependencies as with the new "es6-promise" module. A "grunt copy" task could then move the files to src/compression/

[tanx]

Currently the sources of dependencies are included but I think we want to move away from that and use npm dependencies as with the new "es6-promise" module. A "grunt copy" task could then move the files to src/compression/

We wouldn't even need to copy the files. The browserify build task looks for the common.js modules in the node_modules directory automatically to resolve dependencies. E.g. see here:

https://github.com/openpgpjs/openpgpjs/blob/master/src/openpgp.js#L46

This dependency is simply provided by the package.jsonhere:

https://github.com/openpgpjs/openpgpjs/blob/master/package.json#L49

[toberndo]

We wouldn't even need to copy the files. The browserify build task looks for the common.js
modules in the node_modules directory automatically to resolve dependencies.

Not sure this will work, becaues in the zlib.js package only "main": "./bin/node-zlib.js", is defined, but we require:

Zlib = require('../compression/zlib.min.js'),
RawInflate = require('../compression/rawinflate.min.js'),
RawDeflate = require('../compression/rawdeflate.min.js');

[laktak]

@tanx , @toberndo so should I switch zlib to npm + grunt?

[tanx]

@toberndo I see. Well yes then I guess a grunt copy step is necessary.

@Iaktak that would be good. Then we wouldn't have third party builds in the openpgpjs src.

Thanks for the PR by the way. I'm interested to see how this will effect performance.

[laktak]

I've updated the PR. Please take a look a the grunt task though, I usually do gulp ;)

[tanx]

Could you squash (git rebase -i) the commits so that the libs are not in the git history? Thx

[laktak]

Should be fixed.

[tanx]

Thx!

@toberndo you have more insight into the compression code than me. Could you review the code? I'll be happy to create a new release after that.

[toberndo]

Did some performance testing and the numbers we get are really nice.

Decompression, 5 MB file:

v0.8.2

zip    2391 ms
zlib   3320 ms

with zlib.js

zip    497 ms
zlib   391 ms

I also verified that #55 is fixed with this PR.

Ready to merge.

[tanx]

@toberndo sound great! Thanks for looking at this

I'll merge and release as 0.9

[tanx]

Released v0.9.0 on npm and minified build here: https://github.com/openpgpjs/openpgpjs/releases/tag/v0.9.0

Great job guys!

[planeguy]

Noob question: How do you use this compression for outgoing encrypted messages? openpgp.config.compresson?

[PawelGorny]

Do you have any junit test to compare the performance?

[toberndo]

@planeguy Compression for outgoing encrypted messages is not yet supported in the high level API.

@PawelGorny I pushed my unit test here: mailvelope@6be3e5f

[barrydegraaff]

This sounds good, would it be possible to include a (single file) build of zlib with the release of openpgpjs, so we don't all need to build that ourselves?

The same as been done for the openpgp.worker.js?

[toberndo]

@barrydegraaff zlib is already part of dist/openpgp.js and dist/openpgp.min.js. You just need to fetch it with npm update before doing the build with grunt.

[alexluft]

Absolutely awesome! Is someone working on adding compression support for outgoing messages to high level API? Some kind of config switch?

[puzrin]

I'd recomment to take a look at https://github.com/nodeca/pako for inflate/deflate