You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello, i'm rewriting a Docker logger driver for Gelf format supporting multiline. I have done it in Golang but after some issues with performance (Previous version used compression and tries to parse Json from the log).
I'm trying to bring a support for compression level in Gzip and Zlib, using libdeflater, the only downside is that for each thread for parallel processing needs its own compressor object as described in Support for parallel processing #40
DEFLATE (and zlib and gzip) streams aren't suitable for parallel decompression.
However, if you aren't locked into a data format that uses a single stream, you can easily parallelize at the application layer by dividing the data into chunks before compression, then compressing and/or decompressing the chunks in parallel. libdeflate already works fine for this; just make sure to allocate a separate libdeflate_compressor or libdeflate_decompressor for each concurrent thread.
The text was updated successfully, but these errors were encountered:
Hello, i'm rewriting a Docker logger driver for Gelf format supporting multiline. I have done it in Golang but after some issues with performance (Previous version used compression and tries to parse Json from the log).
I'm trying to bring a support for compression level in Gzip and Zlib, using libdeflater, the only downside is that for each thread for parallel processing needs its own compressor object as described in Support for parallel processing #40
The text was updated successfully, but these errors were encountered: