Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: perform rss/wss-aware cgroups memory usage calculation #3052

Merged
merged 2 commits into from
Mar 2, 2023

Conversation

atzoum
Copy link
Contributor

@atzoum atzoum commented Feb 28, 2023

Description

Previous calculation of memory usage under cgroups was not taking into consideration RSS & WSS memory, it was relying only to memory.usage_in_bytes which can be imprecise under circumstances. This article explains how k8s actually calculates RSS & WSS memory respectively.
With this change, we are now calculating both RSS & WSS values and consider the greatest one to be equal to the actual memory usage.

Note: verified in prousmt environment where this inconsistency was initially observed.

Notion Ticket

Link

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@codecov
Copy link

codecov bot commented Feb 28, 2023

Codecov Report

Patch coverage: 71.69% and project coverage change: +0.43 🎉

Comparison is base (7ca1b0b) 52.68% compared to head (0a97ada) 53.11%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3052      +/-   ##
==========================================
+ Coverage   52.68%   53.11%   +0.43%     
==========================================
  Files         334      332       -2     
  Lines       52206    52350     +144     
==========================================
+ Hits        27505    27806     +301     
+ Misses      23110    22946     -164     
- Partials     1591     1598       +7     
Impacted Files Coverage Δ
utils/mem/mem.go 76.92% <0.00%> (-10.04%) ⬇️
warehouse/admin.go 3.17% <0.00%> (+0.14%) ⬆️
warehouse/api.go 69.89% <0.00%> (+0.20%) ⬆️
warehouse/integrations/bigquery/bigquery.go 2.08% <0.00%> (+<0.01%) ⬆️
warehouse/integrations/testhelper/setup.go 0.00% <0.00%> (ø)
warehouse/upload.go 23.31% <0.00%> (+0.05%) ⬆️
warehouse/utils/utils.go 70.07% <ø> (ø)
warehouse/integrations/redshift/redshift.go 10.34% <26.66%> (+0.49%) ⬆️
warehouse/warehousegrpc.go 70.42% <41.30%> (-9.23%) ⬇️
utils/mem/internal/cgroup/mem.go 71.76% <67.39%> (-10.59%) ⬇️
... and 26 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@atzoum atzoum force-pushed the chore.cgroupsMemfix branch 3 times, most recently from 22392eb to 1c32bd1 Compare March 1, 2023 15:15
utils/mem/internal/cgroup/mem.go Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants