Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jemalloc memory profiling integration #1496

Closed
wants to merge 2 commits into from
Closed

Conversation

efirs
Copy link
Contributor

@efirs efirs commented Mar 13, 2017

This change provides low overhead way to get memory allocation statistics of the running instance.

One can get allocation summary using performance_schema.malloc_stats_summary table. Example:

mysql> select * from performance_schema.malloc_stats_summary;

+------------+------------+------------+------------+------------+-----------+
| ALLOCATED  | ACTIVE     | MAPPED     | RESIDENT   | RETAINED   | METADATA  |
+------------+------------+------------+------------+------------+-----------+
| 4444587952 | 5305044912 | 7284843752 | 5427371072 | 1237219680 | 196446784 |
+------------+------------+------------+------------+------------+-----------+
1 row in set (0.02 sec)

Or more detailed statistics by the allocation size and number of requests:

mysql> select * from performance_schema.malloc_stats;
+-------+------------+------------+---------+-------------+
| TYPE  | ALLOCATED  | NMALLOC    | NDALLOC | NREQUESTS   |
+-------+------------+------------+---------+-------------+
| small | 1981056488 | 1240672368 |       0 | 12119092401 |
| large |  344781832 |  101634376 |       0 |  2921581156 |
| huge  | 1904418320 |         53 |       0 |          53 |
+-------+------------+------------+---------+-------------+
3 rows in set (0.02 sec)

Most detailed view of the state of the memory can be produced by dumping memory profile:

mysql> show variables like '%jemalloc%';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| jemalloc_detected  | ON    |
| jemalloc_profiling | ON    |
+--------------------+-------+
2 rows in set (0.00 sec)

mysql> flush memory profile;
Query OK, 0 rows affected (0.01 sec)

[root@/ #]ls /tmp/jeprof_mysqld*
/tmp/jeprof_mysqld.1.0.170013202213
[root@/ #]jeprof --dot /usr/sbin/mysqld /tmp/jeprof_mysqld.1.0.170013202213 > /tmp/jeprof1.dot
[root@/ #]dot -Tpng /tmp/jeprof1.dot > /tmp/jeprof1.png

Here is an example of allocation graph: https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Leak-Checking

Enabling profiling is a three fold process:

  1. Jemalloc needs to be built with profiling enabled (./autogen.sh --enable-prof ...)
  2. Jemalloc and profiling needs to be loaded on the instance start:
    LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 MALLOC_CONF="prof:true" mysqld &
  3. Jemalloc_profiling option controls runtime state of the profiling(using SET GLOBAL), but so as enabled profiling has very little overhead it can be permanently enabled in my.cnf

Performance schema tables are available as soon as Jemalloc is loaded and mysql_upgrade --upgrade-system-tables has been executed for existing instances, no other configuration is needed.

Evgeniy Firsov added 2 commits March 10, 2017 07:50
Added the following:

  * Two new perfschema tables (MALLOC_STATS_SUMMARY, MALLOC_STATS) can be used
to observer current state of memory during runtime.

  * FLUSH MEMORY PROFILE command can be used to dump current profile to file,
    jeprof tool can be used later to inspect what happened between two profiles.

In order functionality to work mysqld must be started with jemalloc and
profiling enabled, othewise it'll be a noop. Profiling can be enabled using
MALLOC_CONF="prof:true" environment variable.

Signed-off-by: Evgeniy Firsov <firsov@uber.com>
Signed-off-by: Evgeniy Firsov <firsov@uber.com>
@laurynas-biveinis laurynas-biveinis self-assigned this Mar 14, 2017
@laurynas-biveinis
Copy link
Contributor

@alexey2k, FYI

@dutow
Copy link
Contributor

dutow commented Mar 24, 2022

Hello!

I'm closing this pull request as it targets the 5.6 release. If it's still relevant to the 5.7/8.0 branch, please reopen it targeting either of them.

@dutow dutow closed this Mar 24, 2022
@LenzGr
Copy link

LenzGr commented Mar 24, 2023

Looks like this was implemented in #4223

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants