Skip to content
目前prometheus官方的mysqld_exporter插件只支持1对1的监控, 此项目在官方插件代码的基础上进行修改, 支持prometheus动态下发host, 实现了一个mysqld_exporter监控多个mysql。1对多监控。 同时也可以用来监控阿里云aliyun的RDS,做到完美兼容
Go Other
  1. Go 99.4%
  2. Other 0.6%
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
collector
vendor
.my.cnf
.promu.yml
.travis.yml
CHANGELOG.md
CONTRIBUTING.md
Dockerfile
LICENSE
MAINTAINERS.md
Makefile
Makefile.common
NOTICE
README.md
VERSION
docker-compose.yml
example.rules
example.rules.yml
go.mod
mysqld_exporter.go
mysqld_exporter_test.go
test_image.sh

README.md

写在前面

有很多小公司目前都使用了阿里云aliyun的rds数据库, 但是对于rds的监控基本上都是使用的阿里云的云监控系统, 没有自己的监控插件。 为了解决阿里云锁死小公司监控技术的问题,本人基于流行的Prometheus监控系统,在其 监控插件源代码的基础上进行了修改。这个开源mysqld_exporter 的开源项目是其中之一,旨在解决rds以及自建数据库的监控 采集问题,并实现一个exporter监控多个数据库的功能。 本项目和本人另外一个开源项目node_exporter 都是基于官方prometheus 插件直接修改, 特点:

  • 1、基于go语言,性能高
  • 2、稳定性高
  • 3、支持1对多监控,极大的减少了运维成本
  • 4、使用方便,无侵入性

MySQL Server Exporter Build Status

CircleCI Docker Repository on Quay Docker Pulls Go Report Card

Prometheus exporter for MySQL server metrics.

Supported MySQL & MariaDB versions: 5.5 and up.

NOTE: Not all collection methods are supported on MySQL/MariaDB < 5.6

Building and running

Required Grants

CREATE USER 'exporter'@'localhost' IDENTIFIED BY 'XXXXXXXX' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter'@'localhost';

NOTE: It is recommended to set a max connection limit for the user to avoid overloading the server with monitoring scrapes under heavy load. This is not supported on all MySQL/MariaDB versions; for example, MariaDB 10.1 (provided with Ubuntu 18.04) does not support this feature.

Build

make

Running

Running using an environment variable:

export DATA_SOURCE_NAME='user:password@(hostname:3306)/'
./mysqld_exporter <flags>

Running using ~/.my.cnf:

./mysqld_exporter <flags>

Example format for flags for version > 0.10.0:

--collect.auto_increment.columns
--no-collect.auto_increment.columns

Example format for flags for version <= 0.10.0:

-collect.auto_increment.columns
-collect.auto_increment.columns=[true|false]

Collector Flags

Name MySQL Version Description
collect.auto_increment.columns 5.1 Collect auto_increment columns and max values from information_schema.
collect.binlog_size 5.1 Collect the current size of all registered binlog files
collect.engine_innodb_status 5.1 Collect from SHOW ENGINE INNODB STATUS.
collect.engine_tokudb_status 5.6 Collect from SHOW ENGINE TOKUDB STATUS.
collect.global_status 5.1 Collect from SHOW GLOBAL STATUS (Enabled by default)
collect.global_variables 5.1 Collect from SHOW GLOBAL VARIABLES (Enabled by default)
collect.info_schema.clientstats 5.5 If running with userstat=1, set to true to collect client statistics.
collect.info_schema.innodb_metrics 5.6 Collect metrics from information_schema.innodb_metrics.
collect.info_schema.innodb_tablespaces 5.7 Collect metrics from information_schema.innodb_sys_tablespaces.
collect.info_schema.innodb_cmp 5.5 Collect InnoDB compressed tables metrics from information_schema.innodb_cmp.
collect.info_schema.innodb_cmpmem 5.5 Collect InnoDB buffer pool compression metrics from information_schema.innodb_cmpmem.
collect.info_schema.processlist 5.1 Collect thread state counts from information_schema.processlist.
collect.info_schema.processlist.min_time 5.1 Minimum time a thread must be in each state to be counted. (default: 0)
collect.info_schema.query_response_time 5.5 Collect query response time distribution if query_response_time_stats is ON.
collect.info_schema.tables 5.1 Collect metrics from information_schema.tables.
collect.info_schema.tables.databases 5.1 The list of databases to collect table stats for, or '*' for all.
collect.info_schema.tablestats 5.1 If running with userstat=1, set to true to collect table statistics.
collect.info_schema.schemastats 5.1 If running with userstat=1, set to true to collect schema statistics
collect.info_schema.userstats 5.1 If running with userstat=1, set to true to collect user statistics.
collect.perf_schema.eventsstatements 5.6 Collect metrics from performance_schema.events_statements_summary_by_digest.
collect.perf_schema.eventsstatements.digest_text_limit 5.6 Maximum length of the normalized statement text. (default: 120)
collect.perf_schema.eventsstatements.limit 5.6 Limit the number of events statements digests by response time. (default: 250)
collect.perf_schema.eventsstatements.timelimit 5.6 Limit how old the 'last_seen' events statements can be, in seconds. (default: 86400)
collect.perf_schema.eventsstatementssum 5.7 Collect metrics from performance_schema.events_statements_summary_by_digest summed.
collect.perf_schema.eventswaits 5.5 Collect metrics from performance_schema.events_waits_summary_global_by_event_name.
collect.perf_schema.file_events 5.6 Collect metrics from performance_schema.file_summary_by_event_name.
collect.perf_schema.file_instances 5.5 Collect metrics from performance_schema.file_summary_by_instance.
collect.perf_schema.indexiowaits 5.6 Collect metrics from performance_schema.table_io_waits_summary_by_index_usage.
collect.perf_schema.tableiowaits 5.6 Collect metrics from performance_schema.table_io_waits_summary_by_table.
collect.perf_schema.tablelocks 5.6 Collect metrics from performance_schema.table_lock_waits_summary_by_table.
collect.perf_schema.replication_group_member_stats 5.7 Collect metrics from performance_schema.replication_group_member_stats.
collect.perf_schema.replication_applier_status_by_worker 5.7 Collect metrics from performance_schema.replication_applier_status_by_worker.
collect.slave_status 5.1 Collect from SHOW SLAVE STATUS (Enabled by default)
collect.slave_hosts 5.1 Collect from SHOW SLAVE HOSTS
collect.heartbeat 5.1 Collect from heartbeat.
collect.heartbeat.database 5.1 Database from where to collect heartbeat data. (default: heartbeat)
collect.heartbeat.table 5.1 Table from where to collect heartbeat data. (default: heartbeat)

General Flags

Name Description
config.my-cnf Path to .my.cnf file to read MySQL credentials from. (default: ~/.my.cnf)
log.level Logging verbosity (default: info)
exporter.lock_wait_timeout Set a lock_wait_timeout on the connection to avoid long metadata locking. (default: 2 seconds)
exporter.log_slow_filter Add a log_slow_filter to avoid slow query logging of scrapes. NOTE: Not supported by Oracle MySQL.
web.listen-address Address to listen on for web interface and telemetry.
web.telemetry-path Path under which to expose metrics.
version Print the version information.

Setting the MySQL server's data source name

The MySQL server's data source name must be set via the DATA_SOURCE_NAME environment variable. The format of this variable is described at https://github.com/go-sql-driver/mysql#dsn-data-source-name.

Customizing Configuration for a SSL Connection

if The MySQL server supports SSL, you may need to specify a CA truststore to verify the server's chain-of-trust. You may also need to specify a SSL keypair for the client side of the SSL connection. To configure the mysqld exporter to use a custom CA certificate, add the following to the mysql cnf file:

ssl-ca=/path/to/ca/file

To specify the client SSL keypair, add the following to the cnf.

ssl-key=/path/to/ssl/client/key
ssl-cert=/path/to/ssl/client/cert

Customizing the SSL configuration is only supported in the mysql cnf file and is not supported if you set the mysql server's data source name in the environment variable DATA_SOURCE_NAME.

Using Docker

You can deploy this exporter using the prom/mysqld-exporter Docker image.

For example:

docker network create my-mysql-network
docker pull prom/mysqld-exporter

docker run -d \
  -p 9104:9104 \
  --network my-mysql-network  \
  -e DATA_SOURCE_NAME="user:password@(my-mysql-network:3306)/" \
  prom/mysqld-exporter

heartbeat

With collect.heartbeat enabled, mysqld_exporter will scrape replication delay measured by heartbeat mechanisms. Pt-heartbeat is the reference heartbeat implementation supported.

Filtering enabled collectors

The mysqld_exporter will expose all metrics from enabled collectors by default. This is the recommended way to collect metrics to avoid errors when comparing metrics of different families.

For advanced use the mysqld_exporter can be passed an optional list of collectors to filter metrics. The collect[] parameter may be used multiple times. In Prometheus configuration you can use this syntax under the scrape config.

params:
  collect[]:
  - foo
  - bar

This can be useful for having different Prometheus servers collect specific metrics from targets.

Example Rules

There are some sample rules available in example.rules

Example config for mysqld_exporter work with prometheus(v2.5)

Prometheus配置:

  - job_name: mysql-exporter
    scrape_interval: 30s
    static_configs:
      - targets: ['exporter-address:9104']
        labels:
         alias: mysqld-exporter

  - job_name: db-mysql
    scrape_interval: 30s
    consul_sd_configs:
      - server: 'consul-address'
        services: ["db-mysql"]
        tag: "hz-ali"
        tag_separator: "|"

    relabel_configs:
      - source_labels: [__meta_consul_tags]
        target_label: tags
      - source_labels: [__meta_consul_service_metadata_alias]
        target_label: alias
      - source_labels: [__meta_consul_service_metadata_ip]
        target_label: instance
      - source_labels: [__meta_consul_service_metadata_service_type]
        target_label: service_type
      - source_labels: [__meta_consul_service_metadata_service_detail]
        target_label: service_detail

      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: exporter-address:9104

mysqld_exporter配置.my.cnf:

[client]
user=user
password=password123
host=localhost

更多详细内容,参考本人的相关文档: 简书:rds数据库本地化改造
点我达博客:rds数据库本地化改造
微信公众号:rds数据库本地化改造

You can’t perform that action at this time.