Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

box.tuple.new: hidden memory usage #5872

Closed
filonenko-mikhail opened this issue Mar 10, 2021 · 2 comments · Fixed by #7028
Closed

box.tuple.new: hidden memory usage #5872

filonenko-mikhail opened this issue Mar 10, 2021 · 2 comments · Fixed by #7028
Labels
bug Something isn't working srv.info Collection of maintenance tickets about box.info like interfaces

Comments

@filonenko-mikhail
Copy link
Collaborator

Bug description

In case when using box.tuple.new there is no place to understand how many memory is used.

Initial state:

tarantool> box.cfg{}
---
...

tarantool> collectgarbage('collect') collectgarbage('collect')
---
...

tarantool> box.info().memory()
---
- cache: 0
  data: 37824
  index: 1196032
  lua: 1013538
  net: 589824
  tx: 0
...

tarantool> box.slab.info()
---
- items_size: 554560
  items_used_ratio: 6.82%
  quota_size: 268435456
  quota_used_ratio: 12.50%
  arena_used_ratio: 3.7%
  items_used: 37824
  quota_used: 33554432
  arena_size: 33554432
  arena_used: 1233856
...

pmap

pmap -x -p `pidof tarantool` | tail -n2
---------------- ------- ------- ------- 
total kB          796532   57180   40816

"hidden" allocation

r = {} for i = 1, 500 do table.insert(r, box.tuple.new{('x'):rep(1024*1024)}) end

after state

tarantool> box.info().memory()
---
- cache: 0
  data: 37824
  index: 1196032
  lua: 2650930
  net: 589824
  tx: 0
...

tarantool> box.slab.info()
---
- items_size: 554560
  items_used_ratio: 6.82%
  quota_size: 268435456
  quota_used_ratio: 12.50%
  arena_used_ratio: 3.7%
  items_used: 37824
  quota_used: 33554432
  arena_size: 33554432
  arena_used: 1233856
...

the only pmap can show us the memory metrics

pmap -x -p `pidof tarantool` | tail -n2
---------------- ------- ------- ------- 
total kB         1310132  571248  554884

Consider to expose box.tuple pool memory metric

https://gist.github.com/Totktonada/08219ca0cc014a66b83dee6539b80348
@filonenko-mikhail filonenko-mikhail added the bug Something isn't working label Mar 10, 2021
@kyukhin kyukhin added srv.info Collection of maintenance tickets about box.info like interfaces teamC and removed incoming labels Jul 29, 2021
@kyukhin kyukhin added this to the wishlist milestone Aug 19, 2021
@sharonovd
Copy link

This hurts from time to time, I disagree with "wishlisting"

@sharonovd sharonovd removed this from the wishlist milestone Sep 10, 2021
Totktonada added a commit to Totktonada/tarantool that referenced this issue Apr 13, 2022
@Totktonada
lua: track memory occupied by runtime tuples
776af32 
See the documentation request below for details.

Fixes tarantool#5872

@TarantoolBot document
Title: New `box.info.memory().runtime` metric

It allows to track memory allocated for tuples on runtime arena. It does
not count tuples owned by memtx and vinyl, but tracks so called runtime
tuples. The most common example is a tuple created by the
`box.tuple.new(<...>)` function.

Example:

```lua
tarantool> box.info.memory().runtime -- 0
tarantool> box.tuple.new({})
tarantool> box.info.memory().runtime -- 160
tarantool> box.tuple.new({})
tarantool> box.info.memory().runtime -- 320
tarantool> collectgarbage()
tarantool> box.info.memory().runtime -- 160
```
@Totktonada Totktonada mentioned this issue Apr 13, 2022
Merged
Totktonada added a commit to Totktonada/tarantool that referenced this issue Apr 13, 2022
@Totktonada
lua: track memory occupied by runtime tuples
466db71 
See the documentation request below for details.

Fixes tarantool#5872

@TarantoolBot document
Title: New `box.info.memory().runtime.tuple` metric

It allows to track memory allocated for tuples on runtime arena. It does
not count tuples owned by memtx and vinyl, but tracks so called runtime
tuples. The most common example is a tuple created by the
`box.tuple.new(<...>)` function.

Example:

```lua
tarantool> box.info.memory().runtime.tuple -- 0
tarantool> box.tuple.new({})
tarantool> box.info.memory().runtime.tuple -- 160
tarantool> box.tuple.new({})
tarantool> box.info.memory().runtime.tuple -- 320
tarantool> collectgarbage()
tarantool> box.info.memory().runtime.tuple -- 160
```
Totktonada added a commit to Totktonada/tarantool that referenced this issue Apr 14, 2022
@Totktonada
lua: track memory occupied by runtime tuples
ea496f9 
See the documentation request below for details.

Fixes tarantool#5872

@TarantoolBot document
Title: New `box.info.memory().runtime.tuple` metric

It allows to track memory allocated for tuples on runtime arena. It does
not count tuples owned by memtx and vinyl, but tracks so called runtime
tuples. The most common example is a tuple created by the
`box.tuple.new(<...>)` function.

Example:

```lua
tarantool> box.info.memory().runtime.tuple -- 0
tarantool> box.tuple.new({})
tarantool> box.info.memory().runtime.tuple -- 160
tarantool> box.tuple.new({})
tarantool> box.info.memory().runtime.tuple -- 320
tarantool> collectgarbage()
tarantool> box.info.memory().runtime.tuple -- 160
```
Totktonada added a commit to Totktonada/tarantool that referenced this issue Apr 14, 2022
@Totktonada
lua: track memory occupied by runtime tuples
683e8c6 
See the documentation request below for details.

Fixes tarantool#5872

@TarantoolBot document
Title: New `box.runtime.info().tuple` metric

It allows to track memory allocated for tuples on runtime arena. It does
not count tuples owned by memtx and vinyl, but tracks so called runtime
tuples. The most common example is a tuple created by the
`box.tuple.new(<...>)` function.

Example:

```lua
tarantool> box.runtime.info().tuple -- 0
tarantool> box.tuple.new({})
tarantool> box.runtime.info().tuple -- 160
tarantool> box.tuple.new({})
tarantool> box.runtime.info().tuple -- 320
tarantool> collectgarbage()
tarantool> box.runtime.info().tuple -- 160
```
locker pushed a commit that referenced this issue Apr 15, 2022
@Totktonada @locker
lua: track memory occupied by runtime tuples
b56e012 
See the documentation request below for details.

Fixes #5872

@TarantoolBot document
Title: New `box.runtime.info().tuple` metric

It allows to track memory allocated for tuples on runtime arena. It does
not count tuples owned by memtx and vinyl, but tracks so called runtime
tuples. The most common example is a tuple created by the
`box.tuple.new(<...>)` function.

Example:

```lua
tarantool> box.runtime.info().tuple -- 0
tarantool> box.tuple.new({})
tarantool> box.runtime.info().tuple -- 160
tarantool> box.tuple.new({})
tarantool> box.runtime.info().tuple -- 320
tarantool> collectgarbage()
tarantool> box.runtime.info().tuple -- 160
```
@Totktonada Totktonada mentioned this issue Apr 26, 2022
Closed
@Totktonada
Copy link
Member

Documentation issue: tarantool/doc#2829.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working srv.info Collection of maintenance tickets about box.info like interfaces
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

lua: track memory occupied by runtime tuples Totktonada/tarantool
4 participants
@Totktonada @filonenko-mikhail @kyukhin @sharonovd