Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need a more comprehensive max_mem calculation #15

Closed
pulkitsinghal opened this issue May 29, 2012 · 1 comment

Comments

Projects
None yet
1 participant
@pulkitsinghal
Copy link

commented May 29, 2012

On an AWS m1.small instance, the current max_mem calculation of:

max_mem = "#{(node.memory.total.to_i - (node.memory.total.to_i/3) ) / 1024}m"

yields:

rb(main):002:0> ("1697880kB".to_i - ("1697880kB".to_i/3) ) / 1024
=> 1105
irb(main):003:0> 1105*1024
=> 1131520

So -Xmx1105m isn't so bad on a m1.small (1.7GB memory) instance until I run sudo service elasticsearch status -v and see:

elasticsearch 0.19.3 running with PID 1119
...
Memory Total:        1.6gb
...

I know that your setup is perfect for m1.large but I would still like to see a more universal equation than the current one ... so I'll also try and strain my brain to come up with something that factors in permgen etc. before coming up with a max_mem number to use.

Without such a consideration Monit hits the restart if mem_usage > 90% for 15 cycles condition very easily on non m1.large instances like mine.

@pulkitsinghal

This comment has been minimized.

Copy link
Author

commented May 29, 2012

Upon closer inspection of elasticsearch.init.erb, I realized that Memory Total is the machine's total memory and not the total memory being consumed by ES. But the top command really does show 92.6% consumption ... and the following heap_max_in_bytes + non_heap_max_in_bytes ... doesn't add up to more than 80% of the available memory.

$ curl 'http://localhost:9200/_cluster/nodes/elasticsearch-test-1?os&jvm&pretty'
  "jvm" : {
    ...
    "mem" : {
      "heap_init" : "128mb",
      "heap_init_in_bytes" : 134217728,
      "heap_max" : "1gb",
      "heap_max_in_bytes" : 1157562368,
      "non_heap_init" : "23.1mb",
      "non_heap_init_in_bytes" : 24313856,
      "non_heap_max" : "214mb",
      "non_heap_max_in_bytes" : 224395264
    }
  }

Sigh ... sorry for bugging you, I'm closing this until I have a proper suggestion/analysis to present. Too bad that I couldn't get JConsole running on the AWS instance yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.