Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chunk direct buffer usage by networking layer #7811

Closed
wants to merge 3 commits into from

Commits on Sep 21, 2014

  1. Chunk direct buffer usage by networking layer

    Today, due to how netty works (both on http layer and transport layer), and even though the buffers sent over to netty are paged (CompositeChannelBuffer), it ends up re-copying the whole buffer into another heap buffer (bad), and then send it over directly to sun.nio which allocates a full thread local direct buffer to send it (which can be repeated if not all message is sent).
      This is problematic for very large messages, aside from the extra heap temporal usage, the large direct buffers will stay around and not released by the JVM.
      This change forces the use of gathering when building a CompositeChannelBuffer, which results in netty using the sun.nio write method that accepts an array of ByteBuffer (so no extra heap copying), and also reduces the amount of direct memory allocated for large messages.
      See the doc on NettyUtils#DEFAULT_GATHERING for more info.
    closes elastic#7811
    kimchy committed Sep 21, 2014
    Copy the full SHA
    ecf8cee View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2014

  1. fix name of buffer

    kimchy committed Sep 22, 2014
    Copy the full SHA
    23b1c98 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2014

  1. Copy the full SHA
    a24db50 View commit details
    Browse the repository at this point in the history