Skip to content

Commit

Permalink
MDEV-25417: Remove innodb buffer pool load throttling
Browse files Browse the repository at this point in the history
The very lightest of load would decimate any buffer pool loading
to ~1 page per second. As seen in MDEV-29343 this resulting in
a load taking over an hour on a high end system.

Since MDEV-26547 the fetching is asynchronous, however the loading
has equal access to the IO as the SQL queries.
  • Loading branch information
grooverdan authored and dr-m committed Nov 28, 2022
1 parent 812443c commit 183ca82
Showing 1 changed file with 0 additions and 72 deletions.
72 changes: 0 additions & 72 deletions storage/innobase/buf/buf0dump.cc
Original file line number Diff line number Diff line change
Expand Up @@ -410,72 +410,6 @@ buf_dump(
export_vars.innodb_buffer_pool_load_incomplete = 0;
}

/*****************************************************************//**
Artificially delay the buffer pool loading if necessary. The idea of
this function is to prevent hogging the server with IO and slowing down
too much normal client queries. */
UNIV_INLINE
void
buf_load_throttle_if_needed(
/*========================*/
ulint* last_check_time, /*!< in/out: milliseconds since epoch
of the last time we did check if
throttling is needed, we do the check
every srv_io_capacity IO ops. */
ulint* last_activity_count,
ulint n_io) /*!< in: number of IO ops done since
buffer pool load has started */
{
if (n_io % srv_io_capacity < srv_io_capacity - 1) {
return;
}

if (*last_check_time == 0 || *last_activity_count == 0) {
*last_check_time = ut_time_ms();
*last_activity_count = srv_get_activity_count();
return;
}

/* srv_io_capacity IO operations have been performed by buffer pool
load since the last time we were here. */

/* If no other activity, then keep going without any delay. */
if (srv_get_activity_count() == *last_activity_count) {
return;
}

/* There has been other activity, throttle. */

ulint now = ut_time_ms();
ulint elapsed_time = now - *last_check_time;

/* Notice that elapsed_time is not the time for the last
srv_io_capacity IO operations performed by BP load. It is the
time elapsed since the last time we detected that there has been
other activity. This has a small and acceptable deficiency, e.g.:
1. BP load runs and there is no other activity.
2. Other activity occurs, we run N IO operations after that and
enter here (where 0 <= N < srv_io_capacity).
3. last_check_time is very old and we do not sleep at this time, but
only update last_check_time and last_activity_count.
4. We run srv_io_capacity more IO operations and call this function
again.
5. There has been more other activity and thus we enter here.
6. Now last_check_time is recent and we sleep if necessary to prevent
more than srv_io_capacity IO operations per second.
The deficiency is that we could have slept at 3., but for this we
would have to update last_check_time before the
"cur_activity_count == *last_activity_count" check and calling
ut_time_ms() that often may turn out to be too expensive. */

if (elapsed_time < 1000 /* 1 sec (1000 milli secs) */) {
os_thread_sleep((1000 - elapsed_time) * 1000 /* micro secs */);
}

*last_check_time = ut_time_ms();
*last_activity_count = srv_get_activity_count();
}

/*****************************************************************//**
Perform a buffer pool load from the file specified by
innodb_buffer_pool_filename. If any errors occur then the value of
Expand Down Expand Up @@ -622,9 +556,6 @@ buf_load()
std::sort(dump, dump + dump_n);
}

ulint last_check_time = 0;
ulint last_activity_cnt = 0;

/* Avoid calling the expensive fil_space_t::get() for each
page within the same tablespace. dump[] is sorted by (space, page),
so all pages from a given tablespace are consecutive. */
Expand Down Expand Up @@ -699,9 +630,6 @@ buf_load()
return;
}

buf_load_throttle_if_needed(
&last_check_time, &last_activity_cnt, i);

#ifdef UNIV_DEBUG
if ((i+1) >= srv_buf_pool_load_pages_abort) {
buf_load_abort_flag = true;
Expand Down

0 comments on commit 183ca82

Please sign in to comment.