Skip to content
This repository has been archived by the owner on Oct 16, 2020. It is now read-only.

systemd-journald: very high memory usage #2564

Closed
ialidzhikov opened this issue Mar 5, 2019 · 7 comments
Closed

systemd-journald: very high memory usage #2564

ialidzhikov opened this issue Mar 5, 2019 · 7 comments

Comments

@ialidzhikov
Copy link

See #11900.

Currently we use the latest stable release of CoreOS (2023.4.0 by the time of writing this) and we experience high memory usage from systemd-journald side.

  1. What is the eta of the fix as a stable CoreOS release?
  2. What is you proposal for mitigation/temporary solution until 1. is done?
@dm0-
Copy link

dm0- commented Mar 7, 2019

We are planning to release an update with the backported commit on Tuesday. Here's a test build if you want to verify it: http://builds.developer.core-os.net/boards/amd64-usr/2023.4.0%2Bjenkins2-build-2023%2Blocal-2131/coreos_production_image.bin.bz2

@bgilbert
Copy link
Contributor

bgilbert commented Mar 8, 2019

The problem was introduced in Container Linux 2023.0.0.

@ialidzhikov
Copy link
Author

Thank you, really appreciate it. We are waiting for the release on Tuesday.

@ialidzhikov
Copy link
Author

Here is my test - I ran

for i in `seq 1 10000`; do sudo pmap --extended $(pidof systemd-journald) | sort -rnk 2 | head -n 1; done > result.txt

for the latest stable image and the image that you provided in vagrant box (using coreos/coreos-vagrant). You can find below the systemd-journald memory usage before and after the execution of the command.

  • VERSION=2023.4.0

    • First run:
      • Before: 52.1M
      • After: 577.4M
    • Second run:
      • Before: 6.2M
      • After: 606.3M
  • VERSION=2023.4.0+jenkins2-build-2023+local-2131

    • First run:
      • Before: 6.1M
      • After: 38.8M

@bgilbert
Copy link
Contributor

This should be fixed in alpha 2079.0.0, beta 2051.2.0, and stable 2023.5.0, due shortly. Thanks for reporting and testing!

@ialidzhikov
Copy link
Author

Can we already update to 2023.5.0? https://stable.release.core-os.net/amd64-usr/current/ is updated but https://coreos.com/releases/ is not.

@bgilbert
Copy link
Contributor

/current/ is always updated before /releases/ because of the way our tooling works. It's fine to update before release notes are available.

/releases/ is synced now and updates are rolling out.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants