KSM throttling daemon
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ci CI: Fix static-checks script invocation Jun 25, 2018
.github github: Add issue template Apr 27, 2018
pkg signals: Add standard signal handling Jun 7, 2018
trigger
vendor input: Add a virtcontainers based throttling input Nov 16, 2017
.gitignore
.pullapprove.yml CI: Require doc team signoff for doc changes Jan 29, 2018
.travis.yml CI: Remove old test code Jan 31, 2018
CODE_OF_CONDUCT.md CoC: Add Code of Conduct Dec 4, 2017
CONTRIBUTING.md
Gopkg.lock input: Add a virtcontainers based throttling input Nov 16, 2017
Gopkg.toml input: Add a virtcontainers based throttling input Nov 16, 2017
LICENSE Initial commit Nov 15, 2017
Makefile build: Add kata prefix to throttler name Sep 10, 2018
OWNERS
README.md throttler: Make socket path Kata-specific Sep 10, 2018
VERSION release: Kata Containers 1.5.0-rc1 Dec 7, 2018
kata-ksm-throttler.service.in build: Fix systemd unit file build error Sep 11, 2018
kata-vc-throttler.service.in trigger: Make vc throttler Kata-specific Sep 10, 2018
ksm.go license: Bump copyright year Jun 7, 2018
ksm_test.go
throttler.go comments: Don't hard-code generated path Sep 10, 2018
throttler_test.go ksm: Add unit tests Nov 15, 2017

README.md

Build Status Go Report Card Coverage Status GoDoc

KSM throttling daemon

This project implements a Kernel Same-page Merging throttling daemon.

Its goal is to regulate KSM by dynamically modifying the KSM sysfs entries, in order to minimize memory duplication as fast as possible while keeping the KSM daemon load low.

What is KSM?

KSM is a host Linux* kernel feature for de-duplicating memory pages. Although it was initially designed as a KVM specific feature, it is now part of the generic Linux memory management subsystem and can be leveraged by any userspace component or application looking for memory to save.

A daemon (ksmd) periodically scans userspace memory, looking for identical pages that can be replaced by a single, write-protected page. When a process tries to modify this shared page content, it gets a private copy into its memory space. KSM only scans and merges pages that are both anonymous and that have been explictly tagged as mergeable by applications calling into the madvise system call (int madvice(addr, length, MADV_MERGEABLE)).

KSM is customizable through a set of Linux kernel sysfs attributes, the most interesting ones being:

  • /sys/kernel/mm/ksm/run: Turns KSM on (1) and off (0).
  • /sys/kernel/mm/ksm/sleep_millisec: Knob that specifies the KSM scanning period.
  • /sys/kernel/mm/ksm/pages_to_scan: Sets the number of pages KSM will scan per scanning cycle.

The memory density improvements that KSM can provide come at a cost. Depending on the number of anonymous pages it will scan, it can be relatively expensive on CPU utilization.

Overall architecture

This project splits that task into 2 pieces:

  1. The throttling algorithm, implemented as a daemon. The daemon can be asked to throttle KSM up by kicking through its gRPC interface.
  2. The throttling triggers, implemented as gRPC clients.

Daemon

The throttling daemon, ksm-throttler, implements the throttling algorithm on one hand and listens for throttling triggers on the other hand.

Throttling algorithm

By default, ksm-throttler will throttle KSM up and down. Regardless of the current KSM system settings, ksm-throttler will move them to the aggressive settings as soon as it gets triggered. With the aggressive setting, ksmd will run every millisecond and will scan 10% of all available anonymous pages during each scanning cycle.

After switching to the aggressive KSM settings, ksm-throttler will throttle down to the standard setting if it does not get triggered for the next 30 seconds. Then ksm-throttler will continue throttling down to the slow KSM setting if it does not get triggered for the next 2 minutes. Finally, ksm-throttler will get back to the initial KSM settings after two more minutes, unless it gets triggered.

At any point in time, ksm-throttler will get back to to the aggressive setting when getting triggered:

        +----------------+
        |                |
        |    Initial     |
        |    Settings    |<<-------------------------------+
        |                |                                 |
        +-------+--------+                                 |
                |                                          |
                |                                          |
     trigger    |                                          |
                |                                          |
                v                                          |
         +--------------+                                  |
         |  Aggressive  |<<--------+                       |
         +--------------+          |                       |
                |                  |                       |
  No Trigger    |                  |                       |
     (30s)      |                  |    New                |
                |                  |  Trigger              |    No Trigger
                v                  |                       |       (2mn)
         +--------------+          |                       |
         |   Standard   |----------+                       |
         +--------------+          |                       |
                |                  |                       |
  No Trigger    |                  |                       |
     (2mn)      |                  |    New                |
                |                  |  Trigger              |
                v                  |                       |
         +--------------+          |                       |
         |     Slow     |----------+                       |
         +--------------+                                  |
                |                                          |
                |                                          |
                +------------------------------------------+

Throttling triggers

Throttling triggers are gRPC clients to the ksm-throttler daemon. Their role is to identify when KSM needs to be throttled up, depending on which resources they want to monitor.

virtcontainers trigger

This project implements a throttling trigger for virtcontainers based containers, see https://github.com/kata-containers/ksm-throttler/blob/master/trigger/virtcontainers.

gRPC

The current gRPC is very simple, and only consists of a Kick() method:

service KSMThrottler {
	rpc Kick(google.protobuf.Empty) returns (google.protobuf.Empty);
}

A package implements a client API in Go for that interface. For example:

import (
	"flag"
	"fmt"

	"github.com/kata-containers/ksm-throttler/pkg/client"
)

func main() {
	uri := flag.String("uri", "/var/run/kata-ksm-throttler/ksm.sock", "KSM throttler gRPC URI")
	flag.Parse()

	err := client.Kick(*uri)
	if err != nil {
		fmt.Println(err)
	}
}

Build and install

$ make
$ sudo make install

Run

To run ksm-throttler with virtcontainers as the throttling trigger:

$ systemctl start vc-throttler

This will start both the ksm-throttler daemon and the vc throttling trigger.