Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

- #PXC-393: Optimized and accurate way to get mmap file gcache size using mincore syscall #100

Open
wants to merge 1 commit into
base: 3.x
Choose a base branch
from

Conversation

sysprg
Copy link

@sysprg sysprg commented Dec 14, 2016

This patch implements a new information schema table ("WSREP_STATS"),
which contains the "wsrep_gcache_actual_pool_size" variable that reflects
the exact amount of memory, which is used by the "gcache" memory pool.

Using a separate information schema table for this variable avoids
deceleration of regular queries, such as SHOW STATUS.

To determine the actual amount of memory occupied by the gcache,
the mincore() syscall is used.

At the same time, I limited the size of additional vector, which used
by mincore() syscall, by dividing of large memory ranges into relatively
small 512MB chunks, which are passed to mincore() syscall. This introduces
slight performance loss due to additional syscalls, however, these costs
is less than 1%, while only the 128 kilobytes of additional memory is
allocated for mincore()-related vector.

I made sure that the state of gcache pages measured only up to the
currently achieved "watermark", but not around the entire allocated
pool - to accelerate scenarios such as
https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1462674

In addition, measurement of the actual amount of memory occupied
by the gcache avoids long-term locking of the other gcache operations,
except when mutex is aquired by the allocation or deallocation of
overflow pages (in the "Pages" storage). In this case mid-term
locking is possible – until the code that uses memcore() syscall
will stop working. However, while we do not create/free the overflow
pages, a global lock does not happen.

The related PXC patch is located here: percona/percona-xtradb-cluster#368

…sing mincore syscall

This patch implements a new information schema table ("WSREP_STATS"),
which contains the "wsrep_gcache_actual_pool_size" variable that reflects
the exact amount of memory, which is used by the "gcache" memory pool.

Using a separate information schema table for this variable avoids
deceleration of regular queries, such as SHOW STATUS.

To determine the actual amount of memory occupied by the gcache,
the mincore() syscall is used.

At the same time, I limited the size of additional vector, which used
by mincore() syscall, by dividing of large memory ranges into relatively
small 512MB chunks, which are passed to mincore() syscall. This introduces
slight performance loss due to additional syscalls, however, these costs
is less than 1%, while only the 128 kilobytes of additional memory is
allocated for mincore()-related vector.

I made sure that the state of gcache pages measured only up to the
currently achieved "watermark", but not around the entire allocated
pool - to accelerate scenarios such as
https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1462674

In addition, measurement of the actual amount of memory occupied
by the gcache avoids long-term locking of the other gcache operations,
except when mutex is aquired by the allocation or deallocation of
overflow pages (in the "Pages" storage). In this case mid-term
locking is possible – until the code that uses memcore() syscall
will stop working. However, while we do not create/free the overflow
pages, a global lock does not happen.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant