Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pass zram to agents as well #1328

Merged
merged 1 commit into from
May 1, 2024

Conversation

lennart
Copy link
Contributor

@lennart lennart commented Apr 24, 2024

relates to #1326

@rennokki
Copy link

@mysticaltech Not sure if this would fix this too:
image

control_plane_nodepools = [
  {
    name        = "cp-cax11"
    server_type = "cax11"
    location    = "fsn1"
    labels      = []
    taints      = []
    count       = 3
    swap_size   = "4G"
    zram_size   = "4G"
  },
]

agent_nodepools = [
  {
    name        = "arm-x2"
    server_type = "cax21"
    location    = "fsn1"
    labels      = []
    taints      = []
    count       = 1
    swap_size   = "4G"
    zram_size   = "4G"
  },
  {
    name        = "amd64-x2"
    server_type = "cpx21"
    location    = "fsn1"
    labels      = []
    taints      = []
    count       = 1
    swap_size   = "4G"
    zram_size   = "4G"
  },
]

Copy link
Collaborator

@mysticaltech mysticaltech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lennart Thanks for this, appreciate it!

@mysticaltech
Copy link
Collaborator

@rennokki Here are possible explanations, but please next time, that kind of things are best in a dedicated issue.

The memory usage graph you've provided shows that you are significantly exceeding your available memory resources. Let's analyze the situation based on the nodepool configuration you provided:

Nodepools Configuration:

  • Control Plane Nodepools (cp-cax11):

    • RAM per Node: 8 GB
    • Node Count: 3
    • Total RAM: 24 GB
  • Agent Nodepools:

    • arm-x2 (cax21):
      • RAM per Node: 16 GB
      • Node Count: 1
      • Total RAM: 16 GB
    • amd64-x2 (cpx21):
      • RAM per Node: 8 GB
      • Node Count: 1
      • Total RAM: 8 GB

Total Cluster Memory:

  • Total Available RAM: ( 24 , \text{GB} , (\text{cp-cax11}) + 16 , \text{GB} , (\text{cax21}) + 8 , \text{GB} , (\text{cpx21}) ) = 48 GB

Memory Usage Graph Analysis:

  • Total Reserved Memory: 0.23 GB out of 10 GB (2.3%)
  • Total Used Memory: 15 GB out of 11 GB (136.36%)

The discrepancy here is that your memory usage exceeds the capacity your graphs are set to display (10 GB), which is inconsistent with the actual memory capacity from your nodepools (48 GB). This discrepancy could be due to several reasons:

  1. Misconfiguration in Reporting: The monitoring tool or dashboard might not be configured correctly to represent the total memory available across all nodepools. It's possible it's only tracking one subset or an incorrect aggregation.
  2. Overcommitment of Memory: Kubernetes and many cloud-native systems allow overcommitting resources, where you allocate more virtual resources than physically available, relying on the fact that not all resources are used at the same time.
  3. Possible Inclusion of Swap/ZRAM: You mentioned swap and zram sizes, both set at 4 GB across your nodes, but it's not clear if these are being counted in your total memory calculations or usage in the dashboard.

Recommendations:

  • Verify Dashboard Configuration: Ensure that the dashboard aggregates all memory from all nodepools.
  • Check Node Resource Allocation: Confirm that all nodes are online and correctly reporting their resources. Node misconfigurations or failures could lead to incorrect total memory calculations.
  • Monitor and Optimize Usage: If your usage often exceeds physical limits, consider optimizing your applications for memory usage or adding more nodes to distribute the load.

@mysticaltech mysticaltech merged commit d1aea66 into kube-hetzner:master May 1, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants