Skip to content

Latest commit

 

History

History

mysql

MySQL check

MySQL Dashboard

Overview

The MySQL integration tracks the performance of your MySQL instances. It collects metrics related to throughput, connections, errors, and InnoDB metrics.

Enable Database Monitoring (DBM) for enhanced insights into query performance and database health. In addition to the standard integration, Datadog DBM provides query-level metrics, live and historical query snapshots, wait event analysis, database load, and query explain plans.

MySQL version 5.6, 5.7, 8.0, and MariaDB versions 10.5, 10.6, 10.11 and 11.1 are supported.

Setup

This page describes the MySQL Agent standard integration. If you are looking for the Database Monitoring product for MySQL, see Datadog Database Monitoring.

Installation

The MySQL check is included in the Datadog Agent package. No additional installation is needed on your MySQL server.

Prepare MySQL

Note: To install Database Monitoring for MySQL, select your hosting solution in the Database Monitoring documentation for instructions.

Proceed with the following steps in this guide only if you are installing the standard integration alone.

On each MySQL server, create a database user for the Datadog Agent.

The following instructions grant the Agent permission to login from any host using datadog@'%'. You can restrict the datadog user to be allowed to login only from localhost by using datadog@'localhost'. See MySQL Adding Accounts, Assigning Privileges, and Dropping Accounts for more info.

Create the datadog user with the following command:

mysql> CREATE USER 'datadog'@'%' IDENTIFIED BY '<UNIQUEPASSWORD>';
Query OK, 0 rows affected (0.00 sec)

Verify the user was created successfully using the following commands - replace <UNIQUEPASSWORD> with the password you created above:

mysql -u datadog --password=<UNIQUEPASSWORD> -e "show status" | \
grep Uptime && echo -e "\033[0;32mMySQL user - OK\033[0m" || \
echo -e "\033[0;31mCannot connect to MySQL\033[0m"

The Agent needs a few privileges to collect metrics. Grant the datadog user only the following limited privileges.

For MySQL versions 5.6 and 5.7, grant replication client and set max_user_connections with the following command:

mysql> GRANT REPLICATION CLIENT ON *.* TO 'datadog'@'%' WITH MAX_USER_CONNECTIONS 5;
Query OK, 0 rows affected, 1 warning (0.00 sec)

For MySQL 8.0 or greater, grant replication client and set max_user_connections with the following commands:

mysql> GRANT REPLICATION CLIENT ON *.* TO 'datadog'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> ALTER USER 'datadog'@'%' WITH MAX_USER_CONNECTIONS 5;
Query OK, 0 rows affected (0.00 sec)

Grant the datadog user the process privilege:

mysql> GRANT PROCESS ON *.* TO 'datadog'@'%';
Query OK, 0 rows affected (0.00 sec)

Verify the replication client. Replace <UNIQUEPASSWORD> with the password you created above:

mysql -u datadog --password=<UNIQUEPASSWORD> -e "show slave status" && \
echo -e "\033[0;32mMySQL grant - OK\033[0m" || \
echo -e "\033[0;31mMissing REPLICATION CLIENT grant\033[0m"

If enabled, metrics can be collected from the performance_schema database by granting an additional privilege:

mysql> show databases like 'performance_schema';
+-------------------------------+
| Database (performance_schema) |
+-------------------------------+
| performance_schema            |
+-------------------------------+
1 row in set (0.00 sec)

mysql> GRANT SELECT ON performance_schema.* TO 'datadog'@'%';
Query OK, 0 rows affected (0.00 sec)

To collect index metrics, grant the datadog user an additional privilege:

mysql> GRANT SELECT ON mysql.innodb_index_stats TO 'datadog'@'%';
Query OK, 0 rows affected (0.00 sec)

Configuration

Follow the instructions below to configure this check for an Agent running on a host. For containerized environments, see the Docker, Kubernetes, or ECS sections.

Note: For a full list of available configuration options, see the sample mysql.d/conf.yaml.

Host

To configure this check for an Agent running on a host:

Edit the mysql.d/conf.yaml file, in the conf.d/ folder at the root of your Agent's configuration directory to start collecting your MySQL metrics and logs.

For a full list of available configuration options, see the sample mysql.d/conf.yaml.

Metric collection
  • Add this configuration block to your mysql.d/conf.yaml to collect your MySQL metrics:

    init_config:
    
    instances:
      - host: 127.0.0.1
        username: datadog
        password: "<YOUR_CHOSEN_PASSWORD>" # from the CREATE USER step earlier
        port: "<YOUR_MYSQL_PORT>" # e.g. 3306
        options:
          replication: false
          galera_cluster: true
          extra_status_metrics: true
          extra_innodb_metrics: true
          schema_size_metrics: false
          disable_innodb_metrics: false

Note: Wrap your password in single quotes in case a special character is present.

To collect extra_performance_metrics, your MySQL server must have performance_schema enabled - otherwise set extra_performance_metrics to false. For more information on performance_schema, see MySQL Performance Schema Quick Start.

Note: The datadog user should be set up in the MySQL integration configuration as host: 127.0.0.1 instead of localhost. Alternatively, you may also use sock.

Restart the Agent to start sending MySQL metrics to Datadog.

Log collection

Available for Agent versions >6.0

  1. By default MySQL logs everything in /var/log/syslog which requires root access to read. To make the logs more accessible, follow these steps:

    • Edit /etc/mysql/conf.d/mysqld_safe_syslog.cnf and remove or comment the lines.

    • Edit /etc/mysql/my.cnf and add following lines to enable general, error, and slow query logs:

        [mysqld_safe]
        log_error = /var/log/mysql/mysql_error.log
      
        [mysqld]
        general_log = on
        general_log_file = /var/log/mysql/mysql.log
        log_error = /var/log/mysql/mysql_error.log
        slow_query_log = on
        slow_query_log_file = /var/log/mysql/mysql_slow.log
        long_query_time = 2
      
    • Save the file and restart MySQL using following commands: service mysql restart

    • Make sure the Agent has read access on the /var/log/mysql directory and all of the files within. Double-check your logrotate configuration to make sure those files are taken into account and that the permissions are correctly set there as well.

    • In /etc/logrotate.d/mysql-server there should be something similar to:

        /var/log/mysql.log /var/log/mysql/mysql.log /var/log/mysql/mysql_slow.log {
                daily
                rotate 7
                missingok
                create 644 mysql adm
                Compress
        }
      
  2. Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml file:

    logs_enabled: true
  3. Add this configuration block to your mysql.d/conf.yaml file to start collecting your MySQL logs:

    logs:
      - type: file
        path: "<ERROR_LOG_FILE_PATH>"
        source: mysql
        service: "<SERVICE_NAME>"
    
      - type: file
        path: "<SLOW_QUERY_LOG_FILE_PATH>"
        source: mysql
        service: "<SERVICE_NAME>"
        log_processing_rules:
          - type: multi_line
            name: new_slow_query_log_entry
            pattern: "# Time:"
            # If mysqld was started with `--log-short-format`, use:
            # pattern: "# Query_time:"
            # If using mysql version <5.7, use the following rules instead:
            # - type: multi_line
            #   name: new_slow_query_log_entry
            #   pattern: "# Time|# User@Host"
            # - type: exclude_at_match
            #   name: exclude_timestamp_only_line
            #   pattern: "# Time:"
    
      - type: file
        path: "<GENERAL_LOG_FILE_PATH>"
        source: mysql
        service: "<SERVICE_NAME>"
        # For multiline logs, if they start by the date with the format yyyy-mm-dd uncomment the following processing rule
        # log_processing_rules:
        #   - type: multi_line
        #     name: new_log_start_with_date
        #     pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])
        # If the logs start with a date with the format yymmdd but include a timestamp with each new second, rather than with each log, uncomment the following processing rule
        # log_processing_rules:
        #   - type: multi_line
        #     name: new_logs_do_not_always_start_with_timestamp
        #     pattern: \t\t\s*\d+\s+|\d{6}\s+\d{,2}:\d{2}:\d{2}\t\s*\d+\s+

    See the sample mysql.yaml for all available configuration options, including those for custom metrics.

  4. Restart the Agent.

Docker

To configure this check for an Agent running on a container:

Metric collection

Set Autodiscovery Integration Templates as Docker labels on your application container:

LABEL "com.datadoghq.ad.check_names"='["mysql"]'
LABEL "com.datadoghq.ad.init_configs"='[{}]'
LABEL "com.datadoghq.ad.instances"='[{"server": "%%host%%", "username": "datadog","password": "<UNIQUEPASSWORD>"}]'

See Autodiscovery template variables for details on using <UNIQUEPASSWORD> as an environment variable instead of a label.

Log collection

Collecting logs is disabled by default in the Datadog Agent. To enable it, see Docker Log Collection.

Then, set Log Integrations as Docker labels:

LABEL "com.datadoghq.ad.logs"='[{"source":"mysql","service":"mysql"}]'

Kubernetes

To configure this check for an Agent running on Kubernetes:

Metric collection

Set Autodiscovery Integrations Templates as pod annotations on your application container. Alternatively, you can configure templates with a file, configmap, or key-value store.

Annotations v1 (for Datadog Agent < v7.36)

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  annotations:
    ad.datadoghq.com/mysql.check_names: '["mysql"]'
    ad.datadoghq.com/mysql.init_configs: '[{}]'
    ad.datadoghq.com/mysql.instances: |
      [
        {
          "server": "%%host%%", 
          "username": "datadog",
          "password": "<UNIQUEPASSWORD>"
        }
      ]
  labels:
    name: mysql
spec:
  containers:
    - name: mysql

Annotations v2 (for Datadog Agent v7.36+)

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  annotations:
    ad.datadoghq.com/mysql.checks: |
      {
        "mysql": {
          "instances": [
            {
              "server": "%%host%%", 
              "username": "datadog",
              "password": "<UNIQUEPASSWORD>"
            }
          ]
        }
      }
  labels:
    name: mysql
spec:
  containers:
    - name: mysql

See Autodiscovery template variables for details on using <UNIQUEPASSWORD> as an environment variable instead of a label.

Log collection

Collecting logs is disabled by default in the Datadog Agent. To enable it, see Kubernetes Log Collection.

Then, set Log Integrations as pod annotations. Alternatively, you can configure this with a file, configmap, or key-value store.

Annotations v1/v2

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  annotations:
    ad.datadoghq.com/mysql.logs: '[{"source": "mysql", "service": "mysql"}]'
  labels:
    name: mysql

ECS

To configure this check for an Agent running on ECS:

Metric collection

Set Autodiscovery Integrations Templates as Docker labels on your application container:

{
  "containerDefinitions": [{
    "name": "mysql",
    "image": "mysql:latest",
    "dockerLabels": {
      "com.datadoghq.ad.check_names": "[\"mysql\"]",
      "com.datadoghq.ad.init_configs": "[{}]",
      "com.datadoghq.ad.instances": "[{\"server\": \"%%host%%\", \"username\": \"datadog\",\"password\": \"<UNIQUEPASSWORD>\"}]"
    }
  }]
}

See Autodiscovery template variables for details on using <UNIQUEPASSWORD> as an environment variable instead of a label.

Log collection

Available for Agent versions >6.0

Collecting logs is disabled by default in the Datadog Agent. To enable it, see ECS Log Collection.

Then, set Log Integrations as Docker labels:

{
  "containerDefinitions": [{
    "name": "mysql",
    "image": "mysql:latest",
    "dockerLabels": {
      "com.datadoghq.ad.logs": "[{\"source\":\"mysql\",\"service\":\"mysql\"}]"
    }
  }]
}

Validation

Run the Agent's status subcommand and look for mysql under the Checks section.

Data Collected

Metrics

See metadata.csv for a list of metrics provided by this integration.

The check does not collect all metrics by default. Set the following boolean configuration options to true to enable the respective metrics:

extra_status_metrics adds the following metrics:

Metric name Metric type
mysql.binlog.cache_disk_use GAUGE
mysql.binlog.cache_use GAUGE
mysql.performance.handler_commit RATE
mysql.performance.handler_delete RATE
mysql.performance.handler_prepare RATE
mysql.performance.handler_read_first RATE
mysql.performance.handler_read_key RATE
mysql.performance.handler_read_next RATE
mysql.performance.handler_read_prev RATE
mysql.performance.handler_read_rnd RATE
mysql.performance.handler_read_rnd_next RATE
mysql.performance.handler_rollback RATE
mysql.performance.handler_update RATE
mysql.performance.handler_write RATE
mysql.performance.opened_tables RATE
mysql.performance.qcache_total_blocks GAUGE
mysql.performance.qcache_free_blocks GAUGE
mysql.performance.qcache_free_memory GAUGE
mysql.performance.qcache_not_cached RATE
mysql.performance.qcache_queries_in_cache GAUGE
mysql.performance.select_full_join RATE
mysql.performance.select_full_range_join RATE
mysql.performance.select_range RATE
mysql.performance.select_range_check RATE
mysql.performance.select_scan RATE
mysql.performance.sort_merge_passes RATE
mysql.performance.sort_range RATE
mysql.performance.sort_rows RATE
mysql.performance.sort_scan RATE
mysql.performance.table_locks_immediate GAUGE
mysql.performance.table_locks_immediate.rate RATE
mysql.performance.threads_cached GAUGE
mysql.performance.threads_created MONOTONIC

extra_innodb_metrics adds the following metrics:

Metric name Metric type
mysql.innodb.active_transactions GAUGE
mysql.innodb.buffer_pool_data GAUGE
mysql.innodb.buffer_pool_pages_data GAUGE
mysql.innodb.buffer_pool_pages_dirty GAUGE
mysql.innodb.buffer_pool_pages_flushed RATE
mysql.innodb.buffer_pool_pages_free GAUGE
mysql.innodb.buffer_pool_pages_total GAUGE
mysql.innodb.buffer_pool_read_ahead RATE
mysql.innodb.buffer_pool_read_ahead_evicted RATE
mysql.innodb.buffer_pool_read_ahead_rnd GAUGE
mysql.innodb.buffer_pool_wait_free MONOTONIC
mysql.innodb.buffer_pool_write_requests RATE
mysql.innodb.checkpoint_age GAUGE
mysql.innodb.current_transactions GAUGE
mysql.innodb.data_fsyncs RATE
mysql.innodb.data_pending_fsyncs GAUGE
mysql.innodb.data_pending_reads GAUGE
mysql.innodb.data_pending_writes GAUGE
mysql.innodb.data_read RATE
mysql.innodb.data_written RATE
mysql.innodb.dblwr_pages_written RATE
mysql.innodb.dblwr_writes RATE
mysql.innodb.hash_index_cells_total GAUGE
mysql.innodb.hash_index_cells_used GAUGE
mysql.innodb.history_list_length GAUGE
mysql.innodb.ibuf_free_list GAUGE
mysql.innodb.ibuf_merged RATE
mysql.innodb.ibuf_merged_delete_marks RATE
mysql.innodb.ibuf_merged_deletes RATE
mysql.innodb.ibuf_merged_inserts RATE
mysql.innodb.ibuf_merges RATE
mysql.innodb.ibuf_segment_size GAUGE
mysql.innodb.ibuf_size GAUGE
mysql.innodb.lock_structs GAUGE
mysql.innodb.locked_tables GAUGE
mysql.innodb.locked_transactions GAUGE
mysql.innodb.log_waits RATE
mysql.innodb.log_write_requests RATE
mysql.innodb.log_writes RATE
mysql.innodb.lsn_current RATE
mysql.innodb.lsn_flushed RATE
mysql.innodb.lsn_last_checkpoint RATE
mysql.innodb.mem_adaptive_hash GAUGE
mysql.innodb.mem_additional_pool GAUGE
mysql.innodb.mem_dictionary GAUGE
mysql.innodb.mem_file_system GAUGE
mysql.innodb.mem_lock_system GAUGE
mysql.innodb.mem_page_hash GAUGE
mysql.innodb.mem_recovery_system GAUGE
mysql.innodb.mem_thread_hash GAUGE
mysql.innodb.mem_total GAUGE
mysql.innodb.os_file_fsyncs RATE
mysql.innodb.os_file_reads RATE
mysql.innodb.os_file_writes RATE
mysql.innodb.os_log_pending_fsyncs GAUGE
mysql.innodb.os_log_pending_writes GAUGE
mysql.innodb.os_log_written RATE
mysql.innodb.pages_created RATE
mysql.innodb.pages_read RATE
mysql.innodb.pages_written RATE
mysql.innodb.pending_aio_log_ios GAUGE
mysql.innodb.pending_aio_sync_ios GAUGE
mysql.innodb.pending_buffer_pool_flushes GAUGE
mysql.innodb.pending_checkpoint_writes GAUGE
mysql.innodb.pending_ibuf_aio_reads GAUGE
mysql.innodb.pending_log_flushes GAUGE
mysql.innodb.pending_log_writes GAUGE
mysql.innodb.pending_normal_aio_reads GAUGE
mysql.innodb.pending_normal_aio_writes GAUGE
mysql.innodb.queries_inside GAUGE
mysql.innodb.queries_queued GAUGE
mysql.innodb.read_views GAUGE
mysql.innodb.rows_deleted RATE
mysql.innodb.rows_inserted RATE
mysql.innodb.rows_read RATE
mysql.innodb.rows_updated RATE
mysql.innodb.s_lock_os_waits RATE
mysql.innodb.s_lock_spin_rounds RATE
mysql.innodb.s_lock_spin_waits RATE
mysql.innodb.semaphore_wait_time GAUGE
mysql.innodb.semaphore_waits GAUGE
mysql.innodb.tables_in_use GAUGE
mysql.innodb.x_lock_os_waits RATE
mysql.innodb.x_lock_spin_rounds RATE
mysql.innodb.x_lock_spin_waits RATE

extra_performance_metrics adds the following metrics:

Metric name Metric type
mysql.performance.query_run_time.avg GAUGE
mysql.performance.digest_95th_percentile.avg_us GAUGE

schema_size_metrics adds the following metric:

Metric name Metric type
mysql.info.schema.size GAUGE

Events

The MySQL check does not include any events.

Service Checks

See service_checks.json for a list of service checks provided by this integration.

Troubleshooting

Further Reading

Additional helpful documentation, links, and articles: