Skip to content
This repository has been archived by the owner. It is now read-only.

systemd-journald: very high memory usage #2564

Closed
ialidzhikov opened this issue Mar 5, 2019 · 7 comments
Closed

systemd-journald: very high memory usage #2564

ialidzhikov opened this issue Mar 5, 2019 · 7 comments

Comments

@ialidzhikov
Copy link

@ialidzhikov ialidzhikov commented Mar 5, 2019

See #11900.

Currently we use the latest stable release of CoreOS (2023.4.0 by the time of writing this) and we experience high memory usage from systemd-journald side.

  1. What is the eta of the fix as a stable CoreOS release?
  2. What is you proposal for mitigation/temporary solution until 1. is done?
@dm0-
Copy link

@dm0- dm0- commented Mar 7, 2019

We are planning to release an update with the backported commit on Tuesday. Here's a test build if you want to verify it: http://builds.developer.core-os.net/boards/amd64-usr/2023.4.0%2Bjenkins2-build-2023%2Blocal-2131/coreos_production_image.bin.bz2

@bgilbert
Copy link
Member

@bgilbert bgilbert commented Mar 8, 2019

The problem was introduced in Container Linux 2023.0.0.

@ialidzhikov
Copy link
Author

@ialidzhikov ialidzhikov commented Mar 11, 2019

Thank you, really appreciate it. We are waiting for the release on Tuesday.

@ialidzhikov
Copy link
Author

@ialidzhikov ialidzhikov commented Mar 11, 2019

Here is my test - I ran

for i in `seq 1 10000`; do sudo pmap --extended $(pidof systemd-journald) | sort -rnk 2 | head -n 1; done > result.txt

for the latest stable image and the image that you provided in vagrant box (using coreos/coreos-vagrant). You can find below the systemd-journald memory usage before and after the execution of the command.

  • VERSION=2023.4.0

    • First run:
      • Before: 52.1M
      • After: 577.4M
    • Second run:
      • Before: 6.2M
      • After: 606.3M
  • VERSION=2023.4.0+jenkins2-build-2023+local-2131

    • First run:
      • Before: 6.1M
      • After: 38.8M
@bgilbert
Copy link
Member

@bgilbert bgilbert commented Mar 11, 2019

This should be fixed in alpha 2079.0.0, beta 2051.2.0, and stable 2023.5.0, due shortly. Thanks for reporting and testing!

@bgilbert bgilbert closed this Mar 11, 2019
@ialidzhikov
Copy link
Author

@ialidzhikov ialidzhikov commented Mar 12, 2019

Can we already update to 2023.5.0? https://stable.release.core-os.net/amd64-usr/current/ is updated but https://coreos.com/releases/ is not.

@bgilbert
Copy link
Member

@bgilbert bgilbert commented Mar 12, 2019

/current/ is always updated before /releases/ because of the way our tooling works. It's fine to update before release notes are available.

/releases/ is synced now and updates are rolling out.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants