Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

systemd-journald: very high memory usage #2564

Closed
ialidzhikov opened this Issue Mar 5, 2019 · 7 comments

Comments

Projects
None yet
3 participants
@ialidzhikov
Copy link

ialidzhikov commented Mar 5, 2019

See #11900.

Currently we use the latest stable release of CoreOS (2023.4.0 by the time of writing this) and we experience high memory usage from systemd-journald side.

  1. What is the eta of the fix as a stable CoreOS release?
  2. What is you proposal for mitigation/temporary solution until 1. is done?
@dm0-

This comment has been minimized.

Copy link
Member

dm0- commented Mar 7, 2019

We are planning to release an update with the backported commit on Tuesday. Here's a test build if you want to verify it: http://builds.developer.core-os.net/boards/amd64-usr/2023.4.0%2Bjenkins2-build-2023%2Blocal-2131/coreos_production_image.bin.bz2

@bgilbert

This comment has been minimized.

Copy link
Member

bgilbert commented Mar 8, 2019

The problem was introduced in Container Linux 2023.0.0.

@ialidzhikov

This comment has been minimized.

Copy link
Author

ialidzhikov commented Mar 11, 2019

Thank you, really appreciate it. We are waiting for the release on Tuesday.

@ialidzhikov

This comment has been minimized.

Copy link
Author

ialidzhikov commented Mar 11, 2019

Here is my test - I ran

for i in `seq 1 10000`; do sudo pmap --extended $(pidof systemd-journald) | sort -rnk 2 | head -n 1; done > result.txt

for the latest stable image and the image that you provided in vagrant box (using coreos/coreos-vagrant). You can find below the systemd-journald memory usage before and after the execution of the command.

  • VERSION=2023.4.0

    • First run:
      • Before: 52.1M
      • After: 577.4M
    • Second run:
      • Before: 6.2M
      • After: 606.3M
  • VERSION=2023.4.0+jenkins2-build-2023+local-2131

    • First run:
      • Before: 6.1M
      • After: 38.8M
@bgilbert

This comment has been minimized.

Copy link
Member

bgilbert commented Mar 11, 2019

This should be fixed in alpha 2079.0.0, beta 2051.2.0, and stable 2023.5.0, due shortly. Thanks for reporting and testing!

@bgilbert bgilbert closed this Mar 11, 2019

@ialidzhikov

This comment has been minimized.

Copy link
Author

ialidzhikov commented Mar 12, 2019

Can we already update to 2023.5.0? https://stable.release.core-os.net/amd64-usr/current/ is updated but https://coreos.com/releases/ is not.

@bgilbert

This comment has been minimized.

Copy link
Member

bgilbert commented Mar 12, 2019

/current/ is always updated before /releases/ because of the way our tooling works. It's fine to update before release notes are available.

/releases/ is synced now and updates are rolling out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.