Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for simple configurable jvm memory overhead percentage #583

Closed
youngm opened this issue Apr 23, 2018 · 5 comments
Closed

Add support for simple configurable jvm memory overhead percentage #583

youngm opened this issue Apr 23, 2018 · 5 comments
Assignees
Labels
Milestone

Comments

@youngm
Copy link
Contributor

youngm commented Apr 23, 2018

With buildpack 4.x we took a significant step forward in more refined memory configuration. We took all the memory items we could reasonably find that were fixed (metaspace, threads, DirectMemory, etc) and set them at fixed sizes so that more of the memory of a container could stay with the heap. However, we've since discovered that applications in larger containers are now more frequently over running the bounds of the container.

We've been able to identify a major components of this being GC memory which grows with the heap #512 However, this memory setting turned out to be difficult to account for. I suspect there are others as well that may be even harder to track down.

In the meantime I propose we add a simple jvm_overhead config value to the jdk which is a simple percentage which the jbp and memory calculator will use as a catch all in an attempt to account for these random unaccountable jvm memory pools that scale with the size of the heap.

In my fork of the JBP I added an 11%. Though my buildpack fork this problem may have been a bit exaggerated because we set -Xss256k leaving less unused thread overhead to act as a jvm memory overhead buffer.

In my fork we also calculate the 11% off of MEMORY_LIMIT since that was easier. Not sure if this value should be calculated off of heap size or MEMORY_LIMIT. I could go either way.

If this issue is accepted then #512 can also be closed.

@youngm youngm changed the title Add support for simple jvm overhead configurable percentage Add support for simple configurable jvm memory overhead percentage Apr 24, 2018
@nebhale nebhale added the jre label May 2, 2018
@nebhale nebhale added this to the v4.12 milestone May 2, 2018
@nebhale
Copy link
Member

nebhale commented May 2, 2018

@glyn Can you please take this as a task to update the memory calculator to take the argument and when that's available I'll add the functionality to the buildpack itself.

@glyn
Copy link
Contributor

glyn commented May 3, 2018

I think the overhead should be a percentage of MEMORY_LIMIT with a default value of 0. It's a blunt tool, but at least it's easy to understand.

glyn added a commit to glyn/java-buildpack-memory-calculator that referenced this issue May 3, 2018
@glyn
Copy link
Contributor

glyn commented May 3, 2018

Implemented memory calculator -headRoom flag which takes a percentage, e.g. -headRoom=11.

@glyn
Copy link
Contributor

glyn commented May 3, 2018

@nebhale When you've added the functionality to the buildpack itself, please would you remember to update how to configure the buildpack in the memory calculator's README.

@nebhale
Copy link
Member

nebhale commented May 3, 2018

Sounds good. Thanks for the quick work @glyn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants