Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MVCC: memory monitoring #6150

Closed
unera opened this issue Jun 16, 2021 · 4 comments · Fixed by #6933
Closed

MVCC: memory monitoring #6150

unera opened this issue Jun 16, 2021 · 4 comments · Fixed by #6933
Assignees
Labels
feature A new functionality mvcc srv.info Collection of maintenance tickets about box.info like interfaces
Milestone

Comments

@unera
Copy link
Collaborator

unera commented Jun 16, 2021

  1. Tarantool is a high RPS application. So, typical tarantool application will have too many RPS.
  2. MVCC: A transaction uses additional memory since the first conflict and until it is closed.

So if You reflect on the pt 1 and 2, You'll find that there can be a situation:

  • someone's opened a transaction and hasn't closed it for a long time
  • high RPS level brings tarantool to OOM killer

So it would be nice to monitor how much memory MVCC uses.

Min/Max/AVG per txn/total

@unera unera added feature A new functionality mvcc labels Jun 16, 2021
@kyukhin kyukhin added the teamC label Jun 25, 2021
@kyukhin kyukhin added this to the 2.10.1 milestone Jun 25, 2021
@kyukhin kyukhin added the srv.info Collection of maintenance tickets about box.info like interfaces label Jun 25, 2021
@alyapunov alyapunov added the 8sp label Jul 9, 2021
@kyukhin kyukhin added the in design Requires design document label Aug 20, 2021
@kyukhin kyukhin assigned unera and unassigned alyapunov Aug 23, 2021
@alyapunov alyapunov assigned drewdzzz and unassigned unera Sep 6, 2021
@alyapunov alyapunov removed the in design Requires design document label Sep 6, 2021
drewdzzz added a commit that referenced this issue Nov 24, 2021
Currently, histogram keeps maximum over all observations
in order to performance so there is no method to get maximum
over contained observations. So this patch introduces method
that solves this problem.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
There is no information about owner of txn_stmt when it is
being allocated. It makes difficult to track memory
consumption for each transaction. That is why we need to
pass transaction itself to txn_stmt_new() instead of its region.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
An opportunity to call garbage collector of memtx transaction
manager manually allows to understand which garbage cannot be freed.
This knowledge can help us to improve garbage collector. Also this
opportunity makes it easier to test memory manager of memtx transaction manager.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
The problem is memtx_tuple_new() says address of memtx_tuple, while
memtx_tuple_delete() says address of an actual tuple (field of memtx_tuple).
Different addresses in logs may be confusing so this patch fixes the problem.

Close #6634
Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close #6635
Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Since transaction can allocate objects on its own, we need a special memory manager
for transactions that will be a base class for all the memory managers for
transaction managers.

One should not create an instance of tx_memory_manager as an independent unit.
It should be used only as a base class.

Module tx_memory requires to register transaction with a special method before
using tx_memory methods with it.

tx_memory contains proxies for all the region and mempool [de]allocation methods
instead of region_truncate() - the only way to truncate txn's region is to clear it.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Memory manager for memtx transaction manager. Derived from tx_memory_manager.

Implemented methods to track memtx_story objects and tuples referenced by them,
added new statuses of txn allocations special for memtx transaction manager.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
It is difficult to monitor memory consumption for each transcation
when some functions allocating on txn's region do not have any information
about owner of the region. This patch solve the problem.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Adopt memtx transaction manager to new memory manager and enable it.
All the transactions use memtx_tx_memory_manager because currently
is the only transaction manager in tarantool.

One can get statistics from memtx_tx_memory_manager via box.stat.memtx.mvcc.

Closes #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Adopt memtx transaction manager to new memory manager and enable it.
All the transactions use memtx_tx_memory_manager because currently
is the only transaction manager in tarantool.

One can get statistics from memtx_tx_memory_manager via box.stat.memtx.mvcc.

Closes #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Currently, histogram keeps maximum over all observations
in order to performance so there is no method to get maximum
over contained observations. So this patch introduces method
that solves this problem.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
There is no information about owner of txn_stmt when it is
being allocated. It makes difficult to track memory
consumption for each transaction. That is why we need to
pass transaction itself to txn_stmt_new() instead of its region.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
An opportunity to call garbage collector of memtx transaction
manager manually allows to understand which garbage cannot be freed.
This knowledge can help us to improve garbage collector. Also this
opportunity makes it easier to test memory manager of memtx transaction manager.

Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
The problem is memtx_tuple_new() says address of memtx_tuple, while
memtx_tuple_delete() says address of an actual tuple (field of memtx_tuple).
Different addresses in logs may be confusing so this patch fixes the problem.

Close #6634
Part of #6150
drewdzzz added a commit that referenced this issue Nov 24, 2021
Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close #6635
Part of #6150
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 3, 2022
Since transaction can allocate objects on its own, we need to track it
to collect memory statistics of memtx txm. Txn's use only region
allocator so this patch introduces tx_region methods - txns should use
it instead of region methods to collect allocation statistics.

Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
NO_TEST=already covered
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 3, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes tarantool#6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.mvcc() method or use index to access a particular
statistic. The statistics format:

txn:
  user data:
    max: 0
    avg: 0
    total: 0
  system data:
    max: 0
    avg: 0
    total: 0
  statements:
    max: 0
    avg: 0
    total: 0
memtx mvcc:
  tuple stories:
    potentially in read view:
      stories count: 0
      retained tuples count: 0
      stories total: 0
      retained tuples total: 0
    directly used by txn:
      stories count: 0
      retained tuples count: 0
      stories total: 0
      retained tuples total: 0
    used for tracking:
      stories count: 0
      retained tuples count: 0
      stories total: 0
      retained tuples total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  trackers:
    max: 0
    avg: 0
    total: 0
@unera
Copy link
Collaborator Author

unera commented Apr 4, 2022

  1. All these counters are in box.stat.memtx.mvcc, so don't use memtx, mvcc as keyname parts.
  2. Don't use phrases with spaces as key names (tuple stories -> tuples, etc)

@drewdzzz
Copy link
Contributor

drewdzzz commented Apr 6, 2022

Method to get statistics: box.stat.memtx.tx()

  txn:
    statements:
      max: 0
      avg: 0
      total: 0
    user:
      max: 0
      avg: 0
      total: 0
    system:
      max: 0
      avg: 0
      total: 0
  mvcc:
    trackers:
      max: 0
      avg: 0
      total: 0
    conflicts:
      max: 0
      avg: 0
      total: 0
    tuples:
      tracking:
        stories:
          count: 0
          total: 0
        retained:
          count: 0
          total: 0
      used:
        stories:
          count: 0
          total: 0
        retained:
          count: 0
          total: 0
      read_view:
        stories:
          count: 0
          total: 0
        retained:
          count: 0
          total: 0

New format of statistics, @unera, what do you think?

@unera
Copy link
Collaborator Author

unera commented Apr 7, 2022

ok

drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close tarantool#6635
Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
Since transaction can allocate objects on its own, we need to track it
to collect memory statistics of memtx txm. Txn's use only region
allocator so this patch introduces tx_region methods - txns should use
it instead of region methods to collect allocation statistics.

Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
NO_TEST=already covered
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes tarantool#6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.mvcc() method or use index to access a particular
statistic. The statistics format:

txn:
  user data:
    max: 0
    avg: 0
    total: 0
  system data:
    max: 0
    avg: 0
    total: 0
  statements:
    max: 0
    avg: 0
    total: 0
memtx mvcc:
  tuple stories:
    potentially in read view:
      stories count: 0
      retained tuples count: 0
      stories total: 0
      retained tuples total: 0
    directly used by txn:
      stories count: 0
      retained tuples count: 0
      stories total: 0
      retained tuples total: 0
    used for tracking:
      stories count: 0
      retained tuples count: 0
      stories total: 0
      retained tuples total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  trackers:
    max: 0
    avg: 0
    total: 0
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes tarantool#6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.tx() method or use index to access a particular
statistic. The statistics format:

txn:
  statements:
    max: 0
    avg: 0
    total: 0
  user:
    max: 0
    avg: 0
    total: 0
  system:
    max: 0
    avg: 0
    total: 0
mvcc:
  trackers:
    max: 0
    avg: 0
    total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  tuples:
    tracking:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    used:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    read_view:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
There is no information about owner of txn_stmt when it is
being allocated. It makes difficult to track memory
consumption for each transaction. That is why we need to
pass transaction itself to txn_stmt_new() instead of its region.

Part of tarantool#6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
An opportunity to call garbage collector of memtx transaction
manager manually allows to understand which garbage cannot be freed.
This knowledge can help us to improve garbage collector. Also this
opportunity makes it easier to test memtx mvcc memory monitoring.

Part of tarantool#6150

NO_DOC=internal feature
NO_CHANGELOG=internal feature
NO_TEST=internal feature
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
It is difficult to monitor memory consumption for each transcation
when some functions allocating on txn's region do not have any information
about owner of the region. This patch solve the problem.

Part of tarantool#6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close tarantool#6635
Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
Since transaction can allocate objects on its own, we need to track it
to collect memory statistics of memtx txm. Txn's use only region
allocator so this patch introduces tx_region methods - txns should use
it instead of region methods to collect allocation statistics.

Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
NO_TEST=already covered
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 7, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes tarantool#6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.tx() method or use index to access a particular
statistic. The statistics format:

txn:
  statements:
    max: 0
    avg: 0
    total: 0
  user:
    max: 0
    avg: 0
    total: 0
  system:
    max: 0
    avg: 0
    total: 0
mvcc:
  trackers:
    max: 0
    avg: 0
    total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  tuples:
    tracking:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    used:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    read_view:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
drewdzzz added a commit to drewdzzz/tarantool that referenced this issue Apr 8, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes tarantool#6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.tx() method or use index to access a particular
statistic. The statistics format:

txn:
  statements:
    max: 0
    avg: 0
    total: 0
  user:
    max: 0
    avg: 0
    total: 0
  system:
    max: 0
    avg: 0
    total: 0
mvcc:
  trackers:
    max: 0
    avg: 0
    total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  tuples:
    tracking:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    used:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    read_view:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
alyapunov pushed a commit that referenced this issue Apr 11, 2022
There is no information about owner of txn_stmt when it is
being allocated. It makes difficult to track memory
consumption for each transaction. That is why we need to
pass transaction itself to txn_stmt_new() instead of its region.

Part of #6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring
alyapunov pushed a commit that referenced this issue Apr 11, 2022
An opportunity to call garbage collector of memtx transaction
manager manually allows to understand which garbage cannot be freed.
This knowledge can help us to improve garbage collector. Also this
opportunity makes it easier to test memtx mvcc memory monitoring.

Part of #6150

NO_DOC=internal feature
NO_CHANGELOG=internal feature
NO_TEST=internal feature
alyapunov pushed a commit that referenced this issue Apr 11, 2022
It is difficult to monitor memory consumption for each transcation
when some functions allocating on txn's region do not have any information
about owner of the region. This patch solve the problem.

Part of #6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring
alyapunov pushed a commit that referenced this issue Apr 11, 2022
Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close #6635
Part of #6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
alyapunov pushed a commit that referenced this issue Apr 11, 2022
Since transaction can allocate objects on its own, we need to track it
to collect memory statistics of memtx txm. Txn's use only region
allocator so this patch introduces tx_region methods - txns should use
it instead of region methods to collect allocation statistics.

Part of #6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
NO_TEST=already covered
alyapunov pushed a commit that referenced this issue Apr 11, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes #6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.tx() method or use index to access a particular
statistic. The statistics format:

txn:
  statements:
    max: 0
    avg: 0
    total: 0
  user:
    max: 0
    avg: 0
    total: 0
  system:
    max: 0
    avg: 0
    total: 0
mvcc:
  trackers:
    max: 0
    avg: 0
    total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  tuples:
    tracking:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    used:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    read_view:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
CuriousGeorgiy pushed a commit to CuriousGeorgiy/tarantool that referenced this issue Apr 12, 2022
There is no information about owner of txn_stmt when it is
being allocated. It makes difficult to track memory
consumption for each transaction. That is why we need to
pass transaction itself to txn_stmt_new() instead of its region.

Part of tarantool#6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring
CuriousGeorgiy pushed a commit to CuriousGeorgiy/tarantool that referenced this issue Apr 12, 2022
An opportunity to call garbage collector of memtx transaction
manager manually allows to understand which garbage cannot be freed.
This knowledge can help us to improve garbage collector. Also this
opportunity makes it easier to test memtx mvcc memory monitoring.

Part of tarantool#6150

NO_DOC=internal feature
NO_CHANGELOG=internal feature
NO_TEST=internal feature
CuriousGeorgiy pushed a commit to CuriousGeorgiy/tarantool that referenced this issue Apr 12, 2022
It is difficult to monitor memory consumption for each transcation
when some functions allocating on txn's region do not have any information
about owner of the region. This patch solve the problem.

Part of tarantool#6150

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=refactoring
CuriousGeorgiy pushed a commit to CuriousGeorgiy/tarantool that referenced this issue Apr 12, 2022
Since a transaction has been prepared its garbage (produced stories
and replaced tuples) cannot be deleted because they are recognized
as used in read-view even if there are no transactions in read-view.
This problem makes it difficult to test the memory monitoring system,
so this patch solves the problem.

Close tarantool#6635
Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
CuriousGeorgiy pushed a commit to CuriousGeorgiy/tarantool that referenced this issue Apr 12, 2022
Since transaction can allocate objects on its own, we need to track it
to collect memory statistics of memtx txm. Txn's use only region
allocator so this patch introduces tx_region methods - txns should use
it instead of region methods to collect allocation statistics.

Part of tarantool#6150

NO_DOC=no visible changes
NO_CHANGELOG=no visible changes
NO_TEST=already covered
CuriousGeorgiy pushed a commit to CuriousGeorgiy/tarantool that referenced this issue Apr 12, 2022
This patch introduces memtx_tx_region and memtx_tx_mempool:
engineers must use only these proxies to collect statistics.

Also this patch introduces box.stat.memtx.mvcc - the way to
get memtx mvcc memory statistics.

Closes tarantool#6150

@TarantoolBot document
Title: Memtx MVCC memory monitoring

Introduce memtx MVCC memory monitoring. One can get it with
box.stat.memtx.tx() method or use index to access a particular
statistic. The statistics format:

txn:
  statements:
    max: 0
    avg: 0
    total: 0
  user:
    max: 0
    avg: 0
    total: 0
  system:
    max: 0
    avg: 0
    total: 0
mvcc:
  trackers:
    max: 0
    avg: 0
    total: 0
  conflicts:
    max: 0
    avg: 0
    total: 0
  tuples:
    tracking:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    used:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
    read_view:
      stories:
        count: 0
        total: 0
      retained:
        count: 0
        total: 0
CuriousGeorgiy added a commit to CuriousGeorgiy/tarantool that referenced this issue Sep 23, 2022
 test

tarantoolgh-6150 test uses Lua assertions which are incomprehensible when failed
(the only information provided is 'assertion failed!'),
making debugging difficult: replace them with luatest assertions.

NO_CHANGELOG=<code health>
NO_DOC=<code health>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new functionality mvcc srv.info Collection of maintenance tickets about box.info like interfaces
Projects
None yet
4 participants