Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new collector exposing 'ksmd' stats #165

Merged
merged 1 commit into from
Jan 21, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions collector/fixtures/e2e-output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,33 @@ node_forks 26442
# HELP node_intr Total number of interrupts serviced.
# TYPE node_intr counter
node_intr 8.885917e+06
# HELP node_ksmd_full_scans_total ksmd 'full_scans' file.
# TYPE node_ksmd_full_scans_total counter
node_ksmd_full_scans_total 323
# HELP node_ksmd_merge_across_nodes ksmd 'merge_across_nodes' file.
# TYPE node_ksmd_merge_across_nodes gauge
node_ksmd_merge_across_nodes 1
# HELP node_ksmd_pages_shared ksmd 'pages_shared' file.
# TYPE node_ksmd_pages_shared gauge
node_ksmd_pages_shared 1
# HELP node_ksmd_pages_sharing ksmd 'pages_sharing' file.
# TYPE node_ksmd_pages_sharing gauge
node_ksmd_pages_sharing 255
# HELP node_ksmd_pages_to_scan ksmd 'pages_to_scan' file.
# TYPE node_ksmd_pages_to_scan gauge
node_ksmd_pages_to_scan 100
# HELP node_ksmd_pages_unshared ksmd 'pages_unshared' file.
# TYPE node_ksmd_pages_unshared gauge
node_ksmd_pages_unshared 0
# HELP node_ksmd_pages_volatile ksmd 'pages_volatile' file.
# TYPE node_ksmd_pages_volatile gauge
node_ksmd_pages_volatile 0
# HELP node_ksmd_run ksmd 'run' file.
# TYPE node_ksmd_run gauge
node_ksmd_run 1
# HELP node_ksmd_sleep_seconds ksmd 'sleep_millisecs' file.
# TYPE node_ksmd_sleep_seconds gauge
node_ksmd_sleep_seconds 0.02
# HELP node_load1 1m load average.
# TYPE node_load1 gauge
node_load1 0.21
Expand Down
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/full_scans
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
323
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/merge_across_nodes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/pages_shared
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/pages_sharing
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
255
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/pages_to_scan
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
100
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/pages_unshared
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/pages_volatile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/run
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1
1 change: 1 addition & 0 deletions collector/fixtures/sys/kernel/mm/ksm/sleep_millisecs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
20
83 changes: 83 additions & 0 deletions collector/ksmd_linux.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// Copyright 2015 The Prometheus Authors
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// +build !noksmd

package collector

import (
"fmt"
"path"

"github.com/prometheus/client_golang/prometheus"
)

var (
ksmdFiles = []string{"full_scans", "merge_across_nodes", "pages_shared", "pages_sharing",
"pages_to_scan", "pages_unshared", "pages_volatile", "run", "sleep_millisecs"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given this collector already explicitly defines the list of files/metrics to export, I wonder whether we should also go for seconds instead of millisecs here? Wdyt @brian-brazil?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this should be seconds. It looks like we can also convert pages to bytes as this feature doesn't support huge pages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, will rename sleep_millisecs to sleep_seconds. Regarding the conversion of pages to bytes - I'm note sure that this is a good idea. First of all, ksmd operates exclusively on pages. And while having metrics like pages_shared, pages_sharing and so on in bytes look good (one might want to see real memory savings in bytes), converting pages_to_scan (which is a setting, not a metric) to bytes just looks weird and confusing.

If someone wants to have bytes instead of pages, this can easily be done with Prometheus query.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We convert pages to bytes where possible everywhere else, so that the users don't have to figure out how to convert from a multitude of different units to get bytes.

It's probably best to exclude pages_to_scan, that doesn't seem like something useful to export as a metric. We'd normally only expose settings if they were a limit, so that you can calculate how full something is, or something that's common to change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably best to exclude pages_to_scan, that doesn't seem like something useful to export as a metric.

Actually, it is useful. It's constantly being adjusted by ksmtuned daemon (on RHEL-based systems), depending on current memory pressure. So it's useful to know this setting's value to correlate it with CPU usage and rate at which pages are being shared.

)

type ksmdCollector struct {
metricDescs map[string]*prometheus.Desc
}

func init() {
Factories["ksmd"] = NewKsmdCollector
}

func getCanonicalMetricName(filename string) string {
switch filename {
case "full_scans":
return filename + "_total"
case "sleep_millisecs":
return "sleep_seconds"
default:
return filename
}
}

// Takes a prometheus registry and returns a new Collector exposing
// kernel/system statistics.
func NewKsmdCollector() (Collector, error) {
subsystem := "ksmd"
descs := make(map[string]*prometheus.Desc)

for _, n := range ksmdFiles {
descs[n] = prometheus.NewDesc(
prometheus.BuildFQName(Namespace, subsystem, getCanonicalMetricName(n)),
fmt.Sprintf("ksmd '%s' file.", n), nil, nil)
}
return &ksmdCollector{descs}, nil
}

// Expose kernel and system statistics.
func (c *ksmdCollector) Update(ch chan<- prometheus.Metric) (err error) {
for _, n := range ksmdFiles {
val, err := readUintFromFile(sysFilePath(path.Join("kernel/mm/ksm", n)))
if err != nil {
return err
}

t := prometheus.GaugeValue
v := float64(val)
switch n {
case "full_scans":
t = prometheus.CounterValue
case "sleep_millisecs":
v /= 1000
}
ch <- prometheus.MustNewConstMetric(c.metricDescs[n], t, v)
}

return nil
}
1 change: 1 addition & 0 deletions end-to-end-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ collectors=$(cat << COLLECTORS
diskstats
entropy
filefd
ksmd
loadavg
mdadm
meminfo
Expand Down