Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heapster sizing adjustments #22940

Merged
merged 1 commit into from Mar 14, 2016

Conversation

mwielgus
Copy link
Contributor

@mwielgus mwielgus commented Mar 14, 2016

Do not merge yet.

@mwielgus mwielgus added sig/autoscaling do-not-merge labels Mar 14, 2016
@mwielgus mwielgus added this to the v1.2 milestone Mar 14, 2016
@k8s-github-robot k8s-github-robot added the size/M label Mar 14, 2016
@k8s-github-robot
Copy link
Contributor

@k8s-github-robot k8s-github-robot commented Mar 14, 2016

Labelling this PR as size/M

@piosz piosz assigned gmarek and unassigned piosz Mar 14, 2016
@k8s-bot
Copy link

@k8s-bot k8s-bot commented Mar 14, 2016

GCE e2e build/test passed for commit 07d5e67.

{% set num_nodes = pillar.get('num_nodes', -1) -%}
{% if num_nodes >= 0 -%}
{% set heapster_memory = (200 + num_nodes * 3)|string + "Mi" -%}
{% set metrics_memory = (200 + num_nodes * 3)|string + "Mi" -%}
{% set eventer_memory = (200000 + num_nodes * 200)|string + "Ki" -%}
Copy link
Contributor

@gmarek gmarek Mar 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/200000/200 * 1024/

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Mar 14, 2016

GCE e2e build/test passed for commit 798f0bd.

@davidopp
Copy link
Member

@davidopp davidopp commented Mar 14, 2016

please ping this thread when you think it is ready to merge

{% set num_nodes = pillar.get('num_nodes', -1) -%}
{% if num_nodes >= 0 -%}
{% set heapster_memory = (200 + num_nodes * 3)|string + "Mi" -%}
{% set metrics_memory = (200 + num_nodes * 4)|string + "Mi" -%}
Copy link
Member

@vishh vishh Mar 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we increasing the memory requirement for heapster pod as a whole?

Copy link
Contributor Author

@mwielgus mwielgus Mar 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Just in case.
For 1000 node cluster we are able to operate without any sink at 2-2.3gb (with 30k pause and 2k system pods). GKE http output creation consumes +400MB. That brings us to 2.75. If couple events occur at once: scraping, gke scraping and pod relist then the temporary memory consumption can be bigger. So to be on the VERY safe side I decided to go with 4 multpilier.

As we did a lot of noise (=events) during the tests it became apparent that Eventer should also have more memory. List that happens inside of watch consumes LOTS of memory (we will try to get rid of it just after 1.2).

@mwielgus mwielgus removed the do-not-merge label Mar 14, 2016
@mwielgus
Copy link
Contributor Author

@mwielgus mwielgus commented Mar 14, 2016

Also bumping the image to beta2.

k8s-teamcity-mesosphere

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TeamCity OSS :: Kubernetes Mesos :: 4 - Smoke Tests Build 19062 outcome was SUCCESS
Summary: Tests passed: 1, ignored: 262 Build time: 00:05:55

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Mar 14, 2016

GCE e2e build/test passed for commit f5f6a80.

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Mar 14, 2016

GCE e2e build/test passed for commit 6123df9.

@davidopp davidopp added lgtm priority/critical-urgent cherrypick-candidate labels Mar 14, 2016
@davidopp
Copy link
Member

@davidopp davidopp commented Mar 14, 2016

LGTM

@k8s-github-robot
Copy link
Contributor

@k8s-github-robot k8s-github-robot commented Mar 14, 2016

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

{% set num_nodes = pillar.get('num_nodes', -1) -%}
{% if num_nodes >= 0 -%}
{% set heapster_memory = (200 + num_nodes * 3)|string + "Mi" -%}
{% set metrics_memory = (200 + num_nodes * 4)|string + "Mi" -%}
{% set eventer_memory = (200000 + num_nodes * 500)|string + "Ki" -%}
Copy link
Member

@piosz piosz Mar 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/s/200000/200*1024

@k8s-bot
Copy link

@k8s-bot k8s-bot commented Mar 14, 2016

GCE e2e build/test passed for commit 6123df9.

@k8s-github-robot
Copy link
Contributor

@k8s-github-robot k8s-github-robot commented Mar 14, 2016

Automatic merge from submit-queue

k8s-github-robot added a commit that referenced this issue Mar 14, 2016
@k8s-github-robot k8s-github-robot merged commit f899a40 into kubernetes:master Mar 14, 2016
4 of 5 checks passed
@bgrant0607
Copy link
Member

@bgrant0607 bgrant0607 commented Mar 15, 2016

PR description still says "do not merge yet".

@bgrant0607
Copy link
Member

@bgrant0607 bgrant0607 commented Mar 15, 2016

Not good for release-note generation

@bgrant0607 bgrant0607 added the cherry-pick-approved label Mar 15, 2016
@bgrant0607
Copy link
Member

@bgrant0607 bgrant0607 commented Mar 15, 2016

And please ensure all PRs have a description or at least a reference to an issue.

eparis pushed a commit to eparis/kubernetes that referenced this issue Mar 15, 2016
Auto commit by PR queue bot
(cherry picked from commit f899a40)
@k8s-cherrypick-bot
Copy link

@k8s-cherrypick-bot k8s-cherrypick-bot commented Mar 15, 2016

Commit be2ad9e found in the "release-1.2" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this s an error find help to get your PR picked.

@mwielgus
Copy link
Contributor Author

@mwielgus mwielgus commented Mar 15, 2016

@bgrant0607 Please merge it to 1.2.

@piosz
Copy link
Member

@piosz piosz commented Mar 16, 2016

@mwielgus this is already merged to 1.2 branch.

shyamjvs pushed a commit to shyamjvs/kubernetes that referenced this issue Dec 1, 2016
Auto commit by PR queue bot
(cherry picked from commit f899a40)
shouhong pushed a commit to shouhong/kubernetes that referenced this issue Feb 14, 2017
Auto commit by PR queue bot
(cherry picked from commit f899a40)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick-approved lgtm priority/critical-urgent sig/autoscaling size/M
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet