Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libheaptrace: Report self consuming memory #14

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

libheaptrace: Report self consuming memory #14

wants to merge 1 commit into from

Conversation

Bojun-Seo
Copy link
Collaborator

Knowing memory overhead can be important in some cases
So add atomic variable to count size and report it

This patch is draft, not working yet.

@Bojun-Seo Bojun-Seo mentioned this pull request Feb 16, 2023
Open
@Bojun-Seo
Copy link
Collaborator Author

Current patch causes segfault.
But I have no idea between this patch and utils::get_comm_name.

(gdb) r samples/a.out
... skip ...
(gdb) bt
#0  0x00007ffff7e405ce in std::_Rb_tree_decrement(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff7fb7982 in std::_Rb_tree_iterator<std::pair<void* const, object_info_t> >::operator-- (this=<synthetic pointer>) at /usr/include/c++/11/bits/stl_tree.h:302
#2  std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_get_insert_unique_pos (this=this@entry=0x7ffff7fbc500 <addrmap>, __k=@0x55555556d0c0: 0x55555556b090) at /usr/include/c++/11/bits/stl_tree.h:2080
#3  0x00007ffff7fb7a6a in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_get_insert_hint_unique_pos (this=this@entry=0x7ffff7fbc500 <addrmap>, __position=..., __k=@0x55555556d0c0: 0x55555556b090) at /usr/include/c++/11/bits/stl_tree.h:2177
#4  0x00007ffff7fb6c01 in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<void* const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<void* const, object_info_t> >, std::piecewise_construct_t const&, std::tuple<void* const&>&&, std::tuple<>&&) (this=this@entry=0x7ffff7fbc500 <addrmap>, __pos=...) at /usr/include/c++/11/bits/stl_tree.h:2435
#5  0x00007ffff7fb7138 in std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::operator[] (__k=@0x7fffffffd758: 0x55555556b090,
    this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:350
#6  get_object (addr=<optimized out>, addr@entry=0x55555556b090) at /home/worker/heaptrace_pr/stacktrace.cc:411
#7  0x00007ffff7fb2664 in operator delete[] (ptr=0x55555556b090) at /home/worker/heaptrace_pr/libheaptrace.cc:208
#8  0x00007ffff7e8ee82 in std::basic_filebuf<char, std::char_traits<char> >::_M_destroy_internal_buffer() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007ffff7e922e7 in std::basic_filebuf<char, std::char_traits<char> >::close() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007ffff7fb85d2 in std::basic_filebuf<char, std::char_traits<char> >::~basic_filebuf (this=0x7fffffffda70, __in_chrg=<optimized out>) at /usr/include/c++/11/fstream:254
#11 std::basic_ifstream<char, std::char_traits<char> >::~basic_ifstream (this=0x7fffffffda60, __in_chrg=<optimized out>, __vtt_parm=<optimized out>) at /usr/include/c++/11/fstream:605
#12 utils::get_comm_name[abi:cxx11]() () at /home/worker/heaptrace_pr/utils.cc:43
#13 0x00007ffff7fb1fe7 in heaptrace_init () at /home/worker/heaptrace_pr/libheaptrace.cc:78
#14 0x00007ffff7fc947e in ?? () from /lib64/ld-linux-x86-64.so.2
#15 0x00007ffff7fc9568 in ?? () from /lib64/ld-linux-x86-64.so.2
#16 0x00007ffff7fe32ea in ?? () from /lib64/ld-linux-x86-64.so.2
#17 0x0000000000000001 in ?? ()
#18 0x00007fffffffe276 in ?? ()
#19 0x0000000000000000 in ?? ()

@Bojun-Seo
Copy link
Collaborator Author

Bojun-Seo commented Feb 23, 2023 •
edited
Loading

To resolve this (crash) issue, I use C APIs on get_comm_name
Bojun-Seo@61889a3
It works normally with this patch.
sample.c has same problem with: #17
factorial.c runs and reports normally.

But after I apply libheaptrace: Report self consuming memory patch on top.
sample.c has same problem but factorial.c crashes on finalize.
Followings are the status of the commits

3b20fca - Bojun Seo, 7 days ago : libheaptrace: Report self consuming memory - Thu Feb 16 19:02:29 2023 +0900
61889a3 - Bojun Seo, 15 minutes ago : utils: Use C APIs on utils::get_comm_name - Thu Feb 23 13:27:57 2023 +0900
703b2ea - Honggyu Kim, 7 weeks ago : heaptrace: Support mulitple sort keys - Thu Jan 5 20:42:26 2023 +0900
f1a90ab - Honggyu Kim, 6 weeks ago : utils:: Add utils::string_split function - Sat Jan 14 09:20:37 2023 +0900
... skip ...

And run heaptrace on gdb with factorial.out

$ gdb heaptrace
(gdb) r samples/factorial.out
[heaptrace] initialized for /proc/5414/maps (factorial.out)
[heaptrace]   finalized for /proc/5414/maps (factorial.out)
=================================================================
[heaptrace] dump allocation sorted by 'size' for /proc/5414/maps (factorial.out)
=== backtrace #1 === [count/peak: 1/1] [size/peak: 32 bytes/32 bytes] [age: 31.47 us]                                                                                                                                0 [0x7ffff7fb241f] operator new(unsigned long) +0x1f
 1 [0x7ffff7fb88b3] void std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11... +0x70
 2 [0x7ffff7fb869e] ./libheaptrace.so (+0x7ffff7fb869e)
 3 [0x7ffff7fb55e7] ./libheaptrace.so (+0x7ffff7fb55e7)
 4 [0x7ffff7fb1ea5] ./libheaptrace.so (+0x7ffff7fb1ea5)
 5 [0x7ffff7fc924e] /lib64/ld-linux-x86-64.so.2 (+0x7ffff7fc924e)
 6 [0x7ffff7dc1495] /lib/x86_64-linux-gnu/libc.so.6 (+0x7ffff7dc1495)
 7 [0x7ffff7dc1610] on_exit +0

=== backtrace #2 === [count/peak: 1/1] [size/peak: 10 bytes/10 bytes] [age: 80.33 us]
 0 [0x7ffff7fb269f] malloc +0x1f
 1 [0x5555555551c4] fac +0xe
 2 [0x5555555551d8] fac +0x13
 3 [0x5555555551d8] fac +0x13
 4 [0x5555555551d8] fac +0x13
 5 [0x5555555551d8] fac +0x13
 6 [0x5555555551d8] fac +0x13
 7 [0x5555555551d8] fac +0x13

... skip ...

=== backtrace #10 === [count/peak: 2/2] [size/peak: 1 bytes/1 bytes] [age: 103.700 us]
 0 [0x7ffff7fb269f] malloc +0x1f
 1 [0x5555555551a5] fac +0x7
 2 [0x5555555551d8] fac +0x13
 3 [0x5555555551d8] fac +0x13
 4 [0x5555555551d8] fac +0x13
 5 [0x5555555551d8] fac +0x13
 6 [0x5555555551d8] fac +0x13
 7 [0x5555555551d8] fac +0x13

[heaptrace] heap traced num of backtrace : 10
[heaptrace] heap traced allocation size  : 80 bytes
[heaptrace] allocator info (virtual)     : 135.168 KB
[heaptrace] allocator info (resident)    : 89.824 KB
[heaptrace] statm info (VSS/RSS/shared)  : 6.246 MB / 2.56 MB / 1.810 MB
[heaptrace] self consumed (memory overhead)  : 5280
=================================================================

Program received signal SIGSEGV, Segmentation fault.

0x00007ffff7c185f4 in std::_Rb_tree_decrement(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff7c185f4 in std::_Rb_tree_decrement(std::_Rb_tree_node_base*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff7fb79a2 in std::_Rb_tree_iterator<std::pair<void* const, object_info_t> >::operator-- (this=<synthetic pointer>) at /usr/include/c++/11/bits/stl_tree.h:302
#2  std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_get_insert_h
int_unique_pos (this=this@entry=0x7ffff7fbc500 <addrmap>, __position=..., __k=@0x55555556e5e0: 0x55555556cf30) at /usr/include/c++/11/bits/stl_tree.h:2185
#3  0x00007ffff7fb6bb1 in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t
> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<void* const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<void* const, object_info_t> >, std::piecewise_construct_t const&, s
td::tuple<void* const&>&&, std::tuple<>&&) (this=this@entry=0x7ffff7fbc500 <addrmap>, __pos=...) at /usr/include/c++/11/bits/stl_tree.h:2435
#4  0x00007ffff7fb70e8 in std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::operator[] (__k=@0x7fffffffda88: 0x55555556cf30,
    this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:350
#5  get_object (addr=<optimized out>, addr@entry=0x55555556cf30) at /home/worker/heaptrace_pr/stacktrace.cc:413
#6  0x00007ffff7fb2524 in operator delete (ptr=0x55555556cf30) at /home/worker/heaptrace_pr/libheaptrace.cc:191
#7  operator delete (ptr=0x55555556cf30) at /home/worker/heaptrace_pr/libheaptrace.cc:186
#8  0x00007ffff7fb6beb in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > >::deallocate (__t=1, __p=<optimized out>, this=0x7ffff7fbc500 <addrmap>)
    at /usr/include/c++/11/ext/new_allocator.h:145
#9  std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > > >::deallocate (__n=1, __p=<optimized out>, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:496
#10 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_put_node (
    __p=<optimized out>, this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:565
#11 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_drop_node (
    __p=<optimized out>, this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:632
#12 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_emplace_hint
_unique<std::piecewise_construct_t const&, std::tuple<void* const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<void* const, object_info_t> >, std::piecewise_construct_t const&, std::tuple<void* const&
>&&, std::tuple<>&&) (this=this@entry=0x7ffff7fbc500 <addrmap>, __pos=...) at /usr/include/c++/11/bits/stl_tree.h:2440
#13 0x00007ffff7fb70e8 in std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::operator[] (__k=@0x7fffffffdb68: 0x55555556d250,
    this=0x7ffff7fbc500 <addrmap>) at /usr/include/c++/11/bits/stl_tree.h:350
#14 get_object (addr=<optimized out>, addr@entry=0x55555556d250) at /home/worker/heaptrace_pr/stacktrace.cc:413
#15 0x00007ffff7fb2524 in operator delete (ptr=0x55555556d250) at /home/worker/heaptrace_pr/libheaptrace.cc:191
#16 operator delete (ptr=0x55555556d250) at /home/worker/heaptrace_pr/libheaptrace.cc:186
#17 0x00007ffff7fb35db in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > >::deallocate (__t=1, __p=0x55555556d250, this=<optimized out>)
    at /usr/include/c++/11/ext/new_allocator.h:132
#18 std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<void* const, object_info_t> > > >::deallocate (__n=1, __p=0x55555556d250, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:496
#19 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_put_node (
    __p=0x55555556d250, this=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:565
#20 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_drop_node (
    __p=0x55555556d250, this=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:632
#21 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::_M_erase (
    __x=0x55555556d250, this=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:1891
#22 0x00007ffff7fb711f in std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t
> > >::_M_erase (this=<optimized out>, __x=0x55555556cdf0) at /usr/include/c++/11/bits/stl_tree.h:787
#23 std::_Rb_tree<void*, std::pair<void* const, object_info_t>, std::_Select1st<std::pair<void* const, object_info_t> >, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::~_Rb_tree (
    this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/stl_tree.h:984
#24 std::map<void*, object_info_t, std::less<void*>, std::allocator<std::pair<void* const, object_info_t> > >::~map (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/stl_map.h:302
#25 0x00007ffff7dc1a56 in __cxa_finalize () from /lib/x86_64-linux-gnu/libc.so.6
#26 0x00007ffff7fb2377 in __do_global_dtors_aux () from ./libheaptrace.so
#27 0x00007fffffffdd90 in ?? ()

@Bojun-Seo Bojun-Seo force-pushed the mem_overhead branch 2 times, most recently from 0203b93 to d98f9cb Compare February 27, 2023 06:17
@Bojun-Seo
libheaptrace: Report heaptrace self consuming memory overhead
c5c0cc4 
Report heaptrace self consuming memory overhead,
for user who want to know memory overhead.
Allocation requests when hook_guard is set are self allocation.

Signed-off-by: Bojun Seo <bojun.seo@lge.com>
@Bojun-Seo
Copy link
Collaborator Author

Bojun-Seo commented Feb 27, 2023 •
edited
Loading

Now it works properly.

Report heaptrace self consuming memory overhead, for user who want to know memory overhead.
Consider allocation requests when hook_guard is set are self allocation.

Add self consumed at the end of the report.

$ ./heaptrace samples/sample_leak.out
[heaptrace] initialized for /proc/6895/maps (sample_leak.out)
[heaptrace]   finalized for /proc/6895/maps (sample_leak.out)
=================================================================
[heaptrace] dump allocation sorted by 'size' for /proc/6895/maps (sample_leak.out)
=== backtrace #1 === [count/peak: 1/1] [size/peak: 4 bytes/4 bytes] [age: 193.682 us]
 0 [0x7f2638a0859f] malloc +0x1f
 1 [0x55baef5991bf] main +0x5
 2 [0x7f26387fbd90] /lib/x86_64-linux-gnu/libc.so.6 (+0x7f26387fbd90)
 3 [0x7f26387fbe40] __libc_start_main +0x20
 4 [0x55baef5990e5] _start +0x9

[heaptrace] heap traced num of backtrace : 1
[heaptrace] heap traced allocation size  : 4 bytes
[heaptrace] allocator info (virtual)     : 135.168 KB
[heaptrace] allocator info (resident)    : 78.368 KB
[heaptrace] statm info (VSS/RSS/shared)  : 6.250 MB / 3.891 MB / 3.661 MB
[heaptrace] self consumed (mem overhead) : 22.187 KB
=================================================================

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@Bojun-Seo