out_stackdriver: support gzip compression

[CatherineF-dev]

Fixes #7089.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
• Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[CatherineF-dev]

• Configuration example:

     [OUTPUT]
         # https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver
         Name                      stackdriver
         ...
         compress_gzip             1

• Debug output:
  Compression rate is about 95%, from 44562525 bytes to \2141049 bytes.

[edsiper]

quick review:

• rename the new configuration option to compress and support a string value. All output plugins that supports compression uses the same configuration property name, so we don't name it differently, as an example take a look at the Splunk plugin: https://github.com/fluent/fluent-bit/blob/master/plugins/out_splunk/splunk.c#L734-L738

• sign-off your commits (DCO error)

once those are fixed we can do another review

[CatherineF-dev]

cc @qingling128

[CatherineF-dev]

It seems splunk is using ctx->compress_gzip. cc @edsiper

https://github.com/fluent/fluent-bit/blob/master/plugins/out_splunk/splunk.c#L600

[CatherineF-dev]

Hi team, have changed to compress

[CatherineF-dev]

/retest

[CatherineF-dev]

Hi team, it's ready to review.

[patrick-stephens]

Can you link the docs PR?

[CatherineF-dev]

cc @qingling128 Need a LGTM on gzip for out_stackdriver plugin, thanks!

[leonardo-albertovich]

I have already approved this PR, @edsiper will merge it when he's able to.

[CatherineF-dev]

Ping ~

[igorpeshansky]

[Optional] There's nearly identical code in out_http [1]. Should we look into a refactoring? Or defer? A question for @edsiper…

[1] https://github.com/fluent/fluent-bit/blob/master/plugins/out_http/http.c#L137-L152

[igorpeshansky]

Plan to add a safe check for NULL compressed

    if (compressed == FLB_TRUE) {
        flb_free(compressed_payload_buffer);
    }

BTW, flb_free uses free [1], which should already handle NULL properly [2].

[1] https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h#L127
[2] https://linux.die.net/man/3/free

[CatherineF-dev]

Thanks, have updated! @igorpeshansky

Okay, we can remove this safe check for flb_free in future PR.

fluent-bit/plugins/out_stackdriver/stackdriver.c

Lines 62 to 64 in 6621b4b

	if (ptr) {
	flb_free(ptr);
	}

[igorpeshansky]

LGTM

[CatherineF-dev]

cc @edsiper, this PR is ready to be merged

[igorpeshansky]

cc @leonardo-albertovich as well.