gzip compression

[saeltz]

Following the discussion on #911.

This drafts an implementation for gzip compression using https://prometheus.github.io/client_java/src-html/io/prometheus/client/exporter/HTTPServer.html as inspiration.

[saeltz]

@SimunKaracic, please take a look.
The failing test is in the untouched DatadogSpanReporterSpec.

[SimunKaracic]

The datadog part was a flip-flop on github actions, feel free to ignore that.
But the failure in kamon-prometheus looks a bit more sinister:

2020-12-10T09:18:18.0477655Z [info] SunHttpServerSpecSuite:
2020-12-10T09:18:18.0889867Z 09:18:18.087 [pool-1-thread-1] INFO kamon.prometheus.PrometheusReporter - Started the embedded HTTP server on http://0.0.0.0:9095
2020-12-10T09:18:18.1077238Z [info] the embedded sun http server
2020-12-10T09:18:18.1697248Z [info] - should provide no data comment on GET to /metrics when no data loaded yet *** FAILED ***
2020-12-10T09:18:18.1698842Z [info]   java.net.SocketException: Unexpected end of file from server
2020-12-10T09:18:18.1700255Z [info]   at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:851)
2020-12-10T09:18:18.1701790Z [info]   at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
2020-12-10T09:18:18.1776582Z [info]   at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:848)
2020-12-10T09:18:18.1778387Z [info]   at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
2020-12-10T09:18:18.1780380Z [info]   at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
2020-12-10T09:18:18.1791442Z [info]   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
2020-12-10T09:18:18.1793374Z [info]   at java.net.URL.openStream(URL.java:1068)
2020-12-10T09:18:18.1794260Z [info]   at scala.io.Source$.fromURL(Source.scala:141)
2020-12-10T09:18:18.1796293Z [info]   at kamon.prometheus.EmbeddedHttpServerSpecSuite.kamon$prometheus$EmbeddedHttpServerSpecSuite$$httpGetMetrics(EmbeddedHttpServerSpec.scala:80)
2020-12-10T09:18:18.1800512Z [info]   at kamon.prometheus.EmbeddedHttpServerSpecSuite$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(EmbeddedHttpServerSpec.scala:36)

I'll check it out later, but we should probably have an additional test that checks that the gzip compression works.

[SimunKaracic]

You're almost over the finish line!

Please fix that one branch, add a test that validates the compression (you can just do what you did with curl in #911), and we're done here :)

[saeltz]

@SimunKaracic, I made the changes you requested but would need some support with the test.

[SimunKaracic]

It looks good!

Next up, small tests:

1. Assert that we return the correct header in the response (Content-Encoding: gzip)
2. Another test where we send the same request to the endpoint, once with gzip, once without it, and check to see that the response size is different.

How does that sound?

[saeltz]

The test cases sound good. I'm just not sure how tu run them.

At the moment, the tests are using scala.io.Source.fromURL(new URL(s"http://127.0.0.1:$port$endpoint")) which does not allow for content encodings.
What do you suggest to use instead?

[SimunKaracic]

It should be possible to add headers to the URL, with something like this:
https://stackoverflow.com/a/9453655

Feel free to make another method httpGetGzippedMetrics, or rip apart the existing one for your purposes!
I'd rather have more cases than the cleanest code!

[SimunKaracic]

Nice, quick response 🎉

This looks done, and it's definitely going in the next release!
I'll give it one more thorough overview and try to break it with some curling, but I think your job here is done.

If there's any small changes to be made, I'll just push them directly, you can go enjoy your holidays :D