Compression

[Licenser]

Pull request

Description

This unifies the various compression and decompression pre and postprocessors into a single pre and a single postprocessor.

Checklist

• The RFC, if required, has been submitted and approved
• Any user-facing impact of the changes is reflected in docs.tremor.rs
• The code is tested
• Use of unsafe code is reasoned about in a comment
• Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
• The performance impact of the change is measured (see below)

Performance

Rally just restructuring

[mfelsche]

While i like the "unification" aspect, i do not see how it is a win for users to have to type a complex json config structure to chose a compression/decompression algo instead of having handy string shorthands.

[mfelsche]

i don't see how removing all those nice short-hands helps users. Now people need to type the very tedious:

{
  "name": "compress",
  "config": {
    "algorithm": "gzip"
  }
}

instead of just "gzip".

[mfelsche]

Do we want to also consider adding elementwise vs stream compression with making the compress processor require a config?

[Licenser]

As we discussed this morning, it reduces the surface we expose as a core, making it simpler, in favor of moving short-hands into userland with definable pre and post processors giving users the option to add syntax sugar to any degree they wish.

Regarding stream compression, so far all compressors have been element wise compression the PR doesn't look at or consider changing this but it'd be a nice ticket to add and consdier

[mfelsche]

Got it.

Lets also have a ticket for defining/creating pre/postprocessors in troy and then i'm good :)

[Licenser]

Done:
#1541

and for completeness

#1540