Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for gzip content-encoding #776

Merged
merged 6 commits into from
Mar 9, 2022

Conversation

ivan-valkov
Copy link
Contributor

Resolves #775

If an Accept-Encoding:gzip header is present, will compress the output and set the correct Content-Encoding header in the response. Added tests for both the wsgi and asgi implementations.

@ivan-valkov ivan-valkov force-pushed the gzip-support branch 2 times, most recently from 5b2c4c4 to 129f2ff Compare February 17, 2022 15:30
Signed-off-by: ivan-valkov <iv.v.valkov@gmail.com>
Copy link
Member

@csmarchbanks csmarchbanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Generally this looks reasonable though I have not fully tested it yet.

One thing to note, the golang client allows disabling compression as part of the options passed to the handler. I am wondering if we should do something similar here? The reason being, in some cases the cpu usage required to compress a large response can slow everything down too much.

prometheus_client/exposition.py Outdated Show resolved Hide resolved
@ivan-valkov
Copy link
Contributor Author

One thing to note, the golang client allows disabling compression as part of the options passed to the handler. I am wondering if we should do something similar here? The reason being, in some cases the cpu usage required to compress a large response can slow everything down too much.

Yeah, definitely, this makes sense. What would be the preferred semantic? GZIP compression enabled or disabled by default? I am conscious that if we enable it by default, some existing users of the client might see unexpected CPU spikes when using the new version of the client, so maybe we should require enabling GZIP compression explicitly?

Copy link
Member

@csmarchbanks csmarchbanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say let's default to negotiating gzip as most use cases will not see much of a CPU spike, and advanced users can turn it off. This also aligns us with client_golang (enabled by default) and client_java (always enabled).

prometheus_client/exposition.py Outdated Show resolved Hide resolved
Signed-off-by: ivan-valkov <iv.v.valkov@gmail.com>
Signed-off-by: ivan-valkov <iv.v.valkov@gmail.com>
@@ -38,6 +39,13 @@
PYTHON376_OR_NEWER = sys.version_info > (3, 7, 5)


def _get_disable_compression() -> bool:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of an environment variable what would you think of passing an additional argument to make_(w|a)sgi_app? My thought is that most users should have control over creating the apps so an env var is not necessary. We try to only use env vars in places where users might not have access to the code, such as registering metrics.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, makes sense. I suppose we need to expose this option in start_http_server as well?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be nice, though not completely required as start_http_server doesn't need to support all configuration for advanced user.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, yeah, this makes sense. I have updated the PR now and added documentation to the README. PTAL when you get some time. I don't think there is a release notes file we need to update but maybe worth mentioning this change when releasing the next version of the client (:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I plan to review this early next week. I will definitely mention it in the release notes for the next version!

Signed-off-by: ivan-valkov <iv.v.valkov@gmail.com>
Signed-off-by: ivan-valkov <iv.v.valkov@gmail.com>
Signed-off-by: ivan-valkov <iv.v.valkov@gmail.com>
Copy link
Member

@csmarchbanks csmarchbanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks a lot!

@csmarchbanks csmarchbanks merged commit 3c91b3f into prometheus:master Mar 9, 2022
@ivan-valkov ivan-valkov deleted the gzip-support branch March 9, 2022 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for GZIP encoding
3 participants