Skip to content

Conversation

@after-the-sunrise
Copy link
Contributor

Hi @brian-brazil,

Generating servlet output for a large number of metrics seems to be inefficient, since the underlying PrintWriter implementation is generating too many org.eclipse.jetty.io.ByteArrayBuffer instances.

Performance improved by simply wrapping the PrintWriter with BufferedWriter. Microbenchmark (ExampleBenchmark, 10k gauges, 100 iterations, laptop machine) shows that it takes around 2.5 sec with the wrapper, and around 5.5 sec without the wrapper.

Copy link
Contributor

@brian-brazil brian-brazil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've failing CI tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we actually need to run a HTTP server to test this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hasn't been addressed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's required since bottleneck is container-specific. For example, with Jetty 8.1.7 the bottleneck seems to be the ByteArrayBuffer instances. But with Jetty 9.4.27, ByteArrayBuffer is no longer an issue but another part of the code is consuming CPU cycles. In either case, the wrapper patch improved the performance by approximately the same amount.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we at least bind to 0 so we don't clash with any existing listeners?

after-the-sunrise and others added 3 commits March 14, 2020 22:08
Signed-off-by: Takanori Takase <scm@after-sunrise.com>
Signed-off-by: Takanori Takase <scm@after-sunrise.com>
Signed-off-by: Takanori Takase <scm@after-sunrise.com>
Signed-off-by: Takanori Takase <scm@after-sunrise.com>
@brian-brazil brian-brazil merged commit ac5136b into prometheus:master Mar 14, 2020
@brian-brazil
Copy link
Contributor

Thanks!

@after-the-sunrise
Copy link
Contributor Author

Thanks for the review & merge :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants