oomkill

A trivial test docker image to consume memory continuously and generate OOMKill events on Kubernetes.

Warning: use this repository and its docker image densify/memeater for testing purposes only, never in production environments!

Docker image

• Source code
• Docker hub

Usage

For testing of various use-cases, memeater can run in one of three modes:

Main Mode

The main process (pid 1 of the container) consumes memory continuously in a loop, then sleeps a configurable number of seconds and exits (or waits forever and does not exit, depending on an environment variable).

This is achieved by running /home/densify/bin/eatmem.sh, which is the default CMD of the docker image.

Forked Mode

The main process (pid 1 of the container) sleeps a configurable number of seconds, then forks another process which consumes memory continuously in a loop; after the forked process exits (and regardless of its exit code, which is logged), the main process then sleeps a configurable number of seconds and exits.

This is achieved by overriding the docker CMD and running /home/densify/bin/fork-eatmem.sh instead.

Please note that under Kubernetes, forked mode DOES NOT cause a container restart for versions prior to 1.28. See also here.

Forked-Many Mode

The main process (pid 1 of the container) forks a configurable number (NUM_FORKS) of memeater processes with AFTER_LOOP_INTERVAL=forever (i.e. these will consume memory and never exit).

It then runs itself forever and checks for the forked processes (10s intervals). As soon as it detects less processes than NUM_FORKS value, it spawns new process/es to complete their number to NUM_FORKS.

This is achieved by overriding the docker CMD and running /home/densify/bin/forkmany.sh instead.

The comment above concerning container restarts under Kubernetes with versions prior to 1.28 is applicable for forked-many mode as well.

Memory Consumption

The memory consumption loop runs a configurable number of times (default: 1,000). Each iteration generates a new POSIX shell variable, which size is determined by another configuration parameter (the default parameter value - 100,000 - indicates memory consumption of about 600,000 bytes); it then sleeps for a configurable number of microseconds.

To calculate the number of bytes generated by the environment variables (OUTER_SEQ and INNER_SEQ) for each iteration, run this shell script with the requested values as arguments - ./testparams.sh outer_seq inner_seq. If the parameters are good, you'll get output like this (and exit code 0):

$ ./testparams.sh 1450 2530
Begin seq 1 generates 10125 bytes
End seq 1450 generates 10125 bytes
Values are equal

If the parameters render different sizes during the outer sequence (and are therefore not good as the exact size is not predictable), you'll get output like (and exit code 255):

$ ./testparams.sh 10000 1000
Begin seq 1 generates 4005 bytes
End seq 10000 generates 5006 bytes
Values are not equal, choose other parameters

Environment Variables

The configuration parameters are all passed to the shell scripts as environment variables:

Environment Variable	Explanation	Default	Main	Forked	Forked-Many
OUTER_SEQ	memory consumption outer loop range	1,000	✅	✅	✅
INNER_SEQ	memory consumption inner loop range	100,000	✅	✅	✅
LOOP_INTERVAL	memory consumption loop sleep (microseconds)	1,000,000	✅	✅	✅
AFTER_LOOP_INTERVAL	sleep after memory consumption loop (seconds); value of forever prevents the process from exiting	2,600	✅	✅	forever
BEFORE_FORK_INTERVAL	sleep before forking memory consumption process (seconds)	10		✅
AFTER_FORK_INTERVAL	sleep after forked process exits (seconds)	360		✅
NUM_FORKS	number of processes to fork	10			✅

Examples

Deployment

Main container, main mode

Frequent OOM Kills

Slow OOM Kills

Main container, forked mode

Frequent OOM Kills

Slow OOM Kills

Main container, forked-many mode

Frequent OOM Kills

Slow OOM Kills

Pod

Main container, main mode

Main container, forked mode

The following two examples refer to a sidecar container. By sidecar we mean the “classic” sidecar, i.e. containers[n], n > 0 - and not the special-case-init-container lifecycle introduced at Kubernetes 1.29.

Sidecar container, main mode

Sidecar container, forked mode