Skip to content

feature: compression #2082

@sehz

Description

@sehz

With compression on, batch contents are compressed by the producer before sending to SPU. SPU will store and send back compressed batches to consumers. The consumer will decompress batches to the client.

The initial version will use a single algorithm that should be leverage SIMD capability to ensure maximum performance. Later iteration will allow custom compression thru SmartModule.

SmartModule should receive uncompressed content unless custom compression is being used.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions