Skip to content

[broker][monitoring] Support gzip compression when expose metrics data#16197

Closed
tjiuming wants to merge 1 commit intoapache:masterfrom
tjiuming:dev/compress_metrics
Closed

[broker][monitoring] Support gzip compression when expose metrics data#16197
tjiuming wants to merge 1 commit intoapache:masterfrom
tjiuming:dev/compress_metrics

Conversation

@tjiuming
Copy link
Contributor

Motivation

support gzip compression when expose metrics data

Modifications

  1. add a configuration: enableMetricsDataCompression

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@github-actions github-actions bot added the doc-required Your PR changes impact docs and you will update later. label Jun 23, 2022
boolean enableCompress) throws IOException {
try {
if (enableCompress) {
out = new GZIPOutputStream(out);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think GZIPOutputStream will create a lot of garbage on the JVM heap :|

Could we just use org.apache.pulsar.common.compression.CompressionCodecZLib on the buffer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@merlimat But CompressionCodecZLib not in GZIP format, and Prometheus read metrics data in GZIP format:

	if s.gzipr == nil {
		s.buf = bufio.NewReader(resp.Body)
		s.gzipr, err = gzip.NewReader(s.buf)
		if err != nil {
			return "", err
		}
	} else {
		s.buf.Reset(resp.Body)
		if err = s.gzipr.Reset(s.buf); err != nil {
			return "", err
		}
	}

	n, err := io.Copy(w, io.LimitReader(s.gzipr, s.bodySizeLimit))
	s.gzipr.Close()
	if err != nil {
		return "", err
	}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-required Your PR changes impact docs and you will update later.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants