feat(outputs.file): Add compression

[zekena2]

Required for all PRs

• Updated associated README.md.
• Wrote appropriate unit tests.
• Pull request title or commits are in conventional commit format

resolves #12875

[Hipska]

Thanks for implementing this. Could you also add a test for the Init error scenario's?

[Hipska]

Would it be much work to also include support for Gzip with internal.CompressWithGzip?

[zekena2]

For using internal.CompressWithGzip I think it's better to use either the stdlib or the the current library to be able to specify the level. I prefer the current library klauspost as it looks optimized more than the stdlib.

[zekena2]

I think the test failed because Circle-CI had a minor outage.

[Hipska]

Great improvements!

[Hipska]

Only this minor remark

[srebhan]

@zekena2 thanks for the good idea and the PR. I want you to look into the existing encoder implementation instead of reimplementing part of it.

[powersj]

@Hipska Can you add the ready for final review label?

@srebhan needs to +1 this first - I've re-requested a review from him

[zekena2]

Is there any updates on this?

[srebhan]

@zekena2 sorry for this taking so long on my side! I do have some more small comments...

[srebhan]

@zekena2 not sure why you always come back to a special handling of identity but I really think it should be no different to other algorithms. Furthermore, the sample.conf description does not match the internal code, so one of them (or both) are wrong...

[zekena2]

The docs in sample.conf matches the code it's just an error message that's causing the confusion. The supported levels are in this line and they're four levels listed in the flate package here which is then used by gzip and zlib.

[srebhan]

The docs in sample.conf matches the code it's just an error message that's causing the confusion. The supported levels are in this line and they're four levels listed in the flate package here which is then used by gzip and zlib.

Ok let me repeat my initial critique/idea: I don't want to expose the inner-workings of the underlying algorithms! So by passing on those values to the user(!!!) we open-up all possible trouble paths. What if one of the library changes the values? What if we need to switch to another implementation with other values?

Do you remember the beginning where I suggested to have best speed, default and best compression? This would be a suitable abstraction still IMO. You and me are engineers loving to dig into technical details, but there are also users out there that don't want to read the Telegraf config, then try to find out what level 3 stands for requiring them to dig into the code to extract which library we use to read the documentation of that library to check what level 3 means.

So my suggested behavior is:

• if you specify a valid level (0,1,9 for gzip and zstd or 1,3,7,11 for zlib) you get exactly that level
• if you specify a value but it is not valid, you get an error
• if you don't specify a level you get the default of the algorithm

With my code, the plugin implements exactly this. As I said above, instead of the numeric levels I would have loved to just use three values best speed, default and best compression as I think there is no real-world case that needs more. :-)

[srebhan]

Some more clarifications...

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz

[srebhan]

Looks good to me. Thanks @zekena2!