Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic configuration for 32-bit elf architecture #54

Closed
Teraslilja opened this issue Mar 5, 2020 · 42 comments
Closed

Automatic configuration for 32-bit elf architecture #54

Teraslilja opened this issue Mar 5, 2020 · 42 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@Teraslilja
Copy link

I am profiling an application that is tested under a Linux distro that is based on 32-bit (i386) architecture.

The Malt is written assuming that elf object is 64-bit architecture only. It would be more than okay, if the Malt could be automatically configured to the correct 32/64 bitness. Or even dynamically detect 32/64 bitness of the elf object.

I have kind of minimal set of changes that allows the 32-bit Malt to pass Unit tests 100% successfully. Not yet tested beyond unit tests.
patch-malt-32-bits.txt

@svalat svalat self-assigned this Mar 5, 2020
@svalat svalat added the enhancement New feature or request label Mar 5, 2020
@svalat
Copy link
Member

svalat commented Mar 5, 2020

I applied your fix and add another one fixing a bug I had since a long time fir fPIE+ASLR.

I made a short test under debian 32 bit via docker, it should work.

Can you confirm to me so I can close the ticket ?

@svalat svalat added this to the v1.2.0 milestone Mar 5, 2020
@svalat
Copy link
Member

svalat commented Mar 5, 2020

Also in your patch I don't understand why you have this :

-		gblState.malloc = (MallocFuncPtr)dlsym(RTLD_NEXT,"malloc");
+		gblState.malloc = (::MallocFuncPtr)dlsym(RTLD_NEXT,"malloc");

@Teraslilja
Copy link
Author

Teraslilja commented Mar 6, 2020

Also in your patch I don't understand why you have this :

-		gblState.malloc = (MallocFuncPtr)dlsym(RTLD_NEXT,"malloc");
+		gblState.malloc = (::MallocFuncPtr)dlsym(RTLD_NEXT,"malloc");

The compiler is rather old and probably does not recognize that the suitable definitions are the same. Without the double '::' (kind of minimal fix), the following error is generated:

[ 38%] Building CXX object src/lib/wrapper/CMakeFiles/malt-libc-wrapper.dir/AllocWrapper.cpp.o
.../malt/malt-32/src/lib/wrapper/AllocWrapper.cpp: In member function ‘void AllocWrapperGlobal::init()’:
.../malt/malt-32/src/lib/wrapper/AllocWrapper.cpp:401: error: reference to ‘MallocFuncPtr’ is ambiguous
.../malt/malt-32/src/lib/wrapper/AllocWrapper.cpp:70: error: candidates are: typedef void* (* MallocFuncPtr)(size_t)
.../malt/malt-32/src/lib/./profiler/AllocStackProfiler.hpp:119: error: typedef void* (* MALT::MallocFuncPtr)(size_t)
.../malt/malt-32/src/lib/wrapper/AllocWrapper.cpp:401: error: reference to ‘MallocFuncPtr’ is ambiguous
.../malt/malt-32/src/lib/wrapper/AllocWrapper.cpp:70: error: candidates are: typedef void* (* MallocFuncPtr)(size_t)
.../malt/malt-32/src/lib/./profiler/AllocStackProfiler.hpp:119: error: typedef void* (* MALT::MallocFuncPtr)(size_t)
.../malt/malt-32/src/lib/wrapper/AllocWrapper.cpp:401: error: expected `;' before ‘dlsym’
make[2]: *** [src/lib/wrapper/CMakeFiles/malt-libc-wrapper.dir/AllocWrapper.cpp.o] Error 1

@Teraslilja
Copy link
Author

Teraslilja commented Mar 6, 2020

When I also tried to use unwind lib with 32-bit Malt, I noticed that there are also 32/64 bitness issues.
patch-malt-32-bits-v2.txt

@svalat
Copy link
Member

svalat commented Mar 6, 2020

I forgot to test libunwind mode, thanks I will give a look on monday evening.

In the mean time, it should work with backtrace mode, this I tested under a debian 32 bit.

Can you tell me what OS/version you are using so I can try to reproduce in same conditions via docker ?

@Teraslilja
Copy link
Author

Teraslilja commented Mar 6, 2020

I forgot to test libunwind mode, thanks I will give a look on monday evening.

In the mean time, it should work with backtrace mode, this I tested under a debian 32 bit.

Can you tell me what OS/version you are using so I can try to reproduce in same conditions via docker ?

I am using Fedora 9 and compiler version is g++ (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8)

@Teraslilja
Copy link
Author

The current master build fails with the following reasons:

  • MallocFuncPtr vs ::MallocFuncPtr (as mentioned earlier)
  • type ticks is apparently only 32-bits, as value 100G (~2^36) if far too much to fit it
    .../malt.master/src/lib/profiler/AllocStackProfiler.cpp:796: error: integer constant is too large for ‘long’ type

When those are considered, then all unit tests are passed.

How ever, there is one 'interesting' warning during compilation:
[ 60%] Building CXX object src/lib/tools/tests/CMakeFiles/TestProcPagemapReader.dir/TestProcPagemapReader.cpp.o
In file included from .../malt.master/src/lib/tools/tests/TestProcPagemapReader.cpp:12:
.../malt.master/src/lib/./tools/ProcPagemapReader.hpp:26: warning: width of ‘MALT::ProcPageMapEntry::pfn’ exceeds its type

For further 32/64 bitness issues: egrep "lib64|-x86_64" -R . --exclude-dir=build
shows a few hard coded 64-bit directory and/or library names

@svalat svalat pinned this issue Mar 8, 2020
svalat added a commit that referenced this issue Mar 8, 2020
@Teraslilja
Copy link
Author

Teraslilja commented Mar 9, 2020

PR #55 passes unit tests 100% as a 32-bit. How ever, Malt libraries are installed to lib64 directory. Apparently this does not work as expected:
#check for lib/lib64 suffix
get_property(LIB64 GLOBAL PROPERTY FIND_LIBRARY_USE_LIB64_PATHS)
if (LIB64)
set(LIBSUFFIX 64)
else()
set(LIBSUFFIX "")
endif()

@Teraslilja
Copy link
Author

I cannot test the 32-bit version malt with this embedded application, because it seems that I have hit by a bug that is not necessary related to 32/64 bitness:

The application hangs (pretty much immediately), when the Malt is enabled. When I check the threads, there is one suspect (no:3)

(gdb) info threads
21 Thread 0xb11c3b90 (LWP 16165) 0x004a2692 in ?? () from /lib/libpthread.so.0
20 Thread 0xb1bc4b90 (LWP 16164) 0x00573416 in __kernel_vsyscall ()
19 Thread 0xb25c5b90 (LWP 16163) 0x004a2695 in ?? () from /lib/libpthread.so.0
18 Thread 0xb2fc6b90 (LWP 16162) 0x00573416 in __kernel_vsyscall ()
17 Thread 0xb39c7b90 (LWP 16161) 0x00573416 in __kernel_vsyscall ()
16 Thread 0xb43c8b90 (LWP 16160) 0x00573416 in __kernel_vsyscall ()
15 Thread 0xb4dc9b90 (LWP 16159) 0x00573416 in __kernel_vsyscall ()
14 Thread 0xb57cab90 (LWP 16158) 0x00573416 in __kernel_vsyscall ()
13 Thread 0xb61cbb90 (LWP 16157) 0x00573416 in __kernel_vsyscall ()
12 Thread 0xb6bccb90 (LWP 16156) 0x00573416 in __kernel_vsyscall ()
11 Thread 0xb75cdb90 (LWP 16155) 0x00573416 in __kernel_vsyscall ()
10 Thread 0xb7fceb90 (LWP 16154) 0x00573416 in __kernel_vsyscall ()
9 Thread 0x6d5db90 (LWP 16153) 0x00573416 in __kernel_vsyscall ()
8 Thread 0x635cb90 (LWP 16152) 0x00573416 in __kernel_vsyscall ()
7 Thread 0x595bb90 (LWP 16151) 0x00573416 in __kernel_vsyscall ()
6 Thread 0x3234b90 (LWP 16150) 0x00573416 in __kernel_vsyscall ()
5 Thread 0x4f5ab90 (LWP 16149) 0x00573416 in __kernel_vsyscall ()
4 Thread 0x3cd6b90 (LWP 16148) 0x00573416 in __kernel_vsyscall ()
3 Thread 0x23f1b90 (LWP 16147) MALT::Scatter2DValues::resize (this=0x94216f4, x=4241004384, y=40)
at .../malt.master/src/lib/valprof/Scatter2DValues.cpp:121
2 Thread 0x19f0b90 (LWP 16146) 0x00573416 in __kernel_vsyscall ()
1 Thread 0x5365d0 (LWP 16143) 0x004a2697 in ?? () from /lib/libpthread.so.0

The backtrace for the thread no:3 is:
(gdb) bt
#0 MALT::Scatter2DValues::resize (this=0x94216f4, x=4241004384, y=40)
at .../malt.master/src/lib/valprof/Scatter2DValues.cpp:121
#1 0x001823a4 in MALT::Scatter2DValues::push (this=0x94216f4, x=4241004384, y=40)
at .../malt.master/src/lib/valprof/Scatter2DValues.cpp:80
#2 0x00185353 in MALT::AllocStackProfiler::onAllocEvent (this=0x94210c0, ptr=0x94f6540, size=40, userStack=0x9508474,
callStackNode=0x23f1174, doLock=true) at .../malt.master/src/lib/profiler/AllocStackProfiler.cpp:227
#3 0x001855ac in MALT::AllocStackProfiler::onCalloc (this=0x94210c0, ptr=0x94f6540, nmemb=1, size=40,
userStack=0x9508474) at .../malt.master/src/lib/profiler/AllocStackProfiler.cpp:128
#4 0x00191d51 in MALT::LocalAllocStackProfiler::onCalloc (this=0x9508450, res=0x94f6540, nmemb=1, size=40, time=2514)
at .../malt.master/src/lib/profiler/LocalAllocStackProfiler.cpp:97
#5 0x0019e161 in calloc (nmemb=1, size=40) at .../malt.master/src/lib/wrapper/AllocWrapper.cpp:603
... application specific
#9 0x00193f13 in MALT::pthreadWrapperStartRoutine (arg=0x94744d0)
at .../malt.master/src/lib/wrapper/ThreadTracker.cpp:62
#10 0x0049e32f in start_thread (arg=) at pthread_create.c:297
#11 0x0075320e in clone () from /lib/libc.so.6

(gdb) list 104,121
104 /******************* FUNCTION *********************/
105 void Scatter2DValues::resize(size_t x, size_t y)
106 {
107 //reset
108 memset(this->oldValues,0,sizeof(size_t)xAxis.sizeyAxis.size);
109
110 //swap values
111 size_t * tmp = values;
112 values = oldValues;
113 oldValues = tmp;
114
115 //new
116 Scatter2DValuesAxis oldXAxis = xAxis;
117 Scatter2DValuesAxis oldYAxis = yAxis;
118
119 //find max
120 while (xAxis.max <= x)
121 xAxis.max *= 2;
(gdb) print x
$16 = 4241004384
(gdb) print xAxis
$17 = {size = 64, log = false, max = 0}

So the loop get stuck, because 0*2 = 0 < 4241004384 for ever.

Why xAxis.max is zero and not e.g. one or x is as huge as 4241004384, is yet uncertain for me.

@Teraslilja
Copy link
Author

//find max
assert(xAxis.max>0);
size_t oldXMax = xAxis.max;
while (xAxis.max <= x)
  {
  xAxis.max *= 2;
  assert(oldXMax<xAxis.max);
  oldXMax=xAxis.max;
  }

Actually this IS a 32-bit issue. The value 4241004384 (or 0xFCC89760) is so big that when max is doubled, it will be bigger than SIZE_MAX or size_t cannot hold without losing the most significant bit. Eventually all significant bits are lost => xAxis.max becomes zero.

And this is also an application bug, the application should not (?) allocate so much memory...

@svalat
Copy link
Member

svalat commented Mar 10, 2020

Thanks for reporting, this is a type issue.

The time axis is in cpu cycles (ticks) which quickly reach big values. On all arch this is a unsigned long long but I copy it into a size_t which is not the good type. I will make a fix as soon as possible.

@Teraslilja
Copy link
Author

Commit fe72063 seems not change the shared library directory name from lib64 to lib at Fedora 9.

-- Installing: /opt/malt/lib64/libmalt.so.1.2.0-dev
-- Installing: /opt/malt/lib64/libmalt.so.1
-- Installing: /opt/malt/lib64/libmalt.so
-- Installing: /opt/malt/lib64/libmalt-base.so.1.2.0-dev
-- Installing: /opt/malt/lib64/libmalt-base.so.1
-- Installing: /opt/malt/lib64/libmalt-base.so
-- Installing: /opt/malt/lib64/libmalt-controler.so.1.2.0-dev
-- Installing: /opt/malt/lib64/libmalt-controler.so.1
-- Installing: /opt/malt/lib64/libmalt-controler.so

@svalat
Copy link
Member

svalat commented Mar 11, 2020

You took the branch, because I didn't yet merge it in master because I just noticed that old cmake does not provide this script. You installed a more recent one on your Fedora 9 of use the 2.4 provided by system ?

@Teraslilja
Copy link
Author

You took the branch, because I didn't yet merge it in master because I just noticed that old cmake does not provide this script. You installed a more recent one on your Fedora 9 of use the 2.4 provided by system ?

I installed cmake 2.8.8 as the minimum requirement required by Malt.

@Teraslilja
Copy link
Author

I changed the type ticks to uint64_t, and now the 32-bit Malt works reasonably well now.

However, there are still two problems here:

  1. no stack information at malt-webview, even thought so configured and instrumented. Could it be so that Malt does not find the configuration from the current working directory (not the same as directory of launched application)? The libmalt.so (at /opt/malt/lib...) object is included in the linking phase

[stack]
enabled=true ; enable stack profiles
mode=enter-exit ; select stack tracing mode (backtrace|enter-exit)
resolve=true ; Automatically resolve symbols with addr2line at exit.
libunwind=true ; Enable of disable usage of libunwind to backtrace.
skip=4 ; Number of stack frame to skip in order to cut at malloc level

  1. Randomly this assert happens quite early after Malt has been activated
    .../malt.master/src/lib/core/SegmentTracker.cpp:48: MALT::SegmentInfo* MALT::SegmentTracker::add(void*, size_t, MALT::MMCallStackNode): ...

Program received signal SIGABRT, Aborted.
...
(gdb) bt
#0 0x00f63416 in __kernel_vsyscall ()
#1 0x00b6b660 in raise (sig=)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2 0x00b6d028 in abort () at abort.c:88
#3 0x00b6457e in __assert_fail (assertion=,
file=, line=,
function=) at assert.c:78
#4 0x00169fca in MALT::SegmentTracker::add (this=0xb0de110, ptr=0xb145370,
size=40, callStack={stack = 0x80cad4, infos = 0x80cb2c})
at .../malt.master/src/lib/core/SegmentTracker.cpp:48
#5 0x0018556f in MALT::AllocStackProfiler::onAllocEvent (this=0xb0de0c0,
ptr=0xb145370, size=40, userStack=0xb161c7c, callStackNode=0x4074174,
doLock=true)
at .../malt.master/src/lib/profiler/AllocStackProfiler.cpp:252
#6 0x00185724 in MALT::AllocStackProfiler::onCalloc (this=0xb0de0c0,
ptr=0xb145370, nmemb=1, size=40, userStack=0xb161c7c)
at .../malt.master/src/lib/profiler/AllocStackProfiler.cpp:128
#7 0x00191ec9 in MALT::LocalAllocStackProfiler::onCalloc (this=0xb161c58,
res=0xb145370, nmemb=1, size=40, time=3176)
at .../malt.master/src/lib/profiler/LocalAllocStackProfiler.cpp:97
#8 0x0019e2d9 in calloc (nmemb=1, size=40)
at .../malt.master/src/lib/wrapper/AllocWrapper.cpp:603
... application specific
#12 0x0019408b in MALT::pthreadWrapperStartRoutine (arg=0xb1314e0)
at .../malt.master/src/lib/wrapper/ThreadTracker.cpp:62
#13 0x00ce432f in start_thread (arg=)
at pthread_create.c:297
#14 0x00c1f20e in clone () from /lib/libc.so.6
(gdb) up
#1 0x00b6b660 in raise (sig=)
at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
Current language: auto; currently c
(gdb) up
#2 0x00b6d028 in abort () at abort.c:88
88 raise (SIGABRT);
(gdb) up
#3 0x00b6457e in __assert_fail (assertion=,
file=, line=,
function=) at assert.c:78
78 abort ();
(gdb) up
#4 0x00169fca in MALT::SegmentTracker::add (this=0xb0de110, ptr=0xb145370,
size=40, callStack={stack = 0x80cad4, infos = 0x80cb2c})
at .../malt.master/src/lib/core/SegmentTracker.cpp:48
48 assert(this->get(ptr) == NULL);
Current language: auto; currently c++
(gdb) list 44,48
44 /******************* FUNCTION *******************/
45 SegmentInfo
SegmentTracker::add(void
ptr, size_t size, MALT::MMCallStackNode callStack)
46 {
47 //check errors
48 assert(this->get(ptr) == NULL);
(gdb)

The both issues are stack related, any possibility that they are related to each other?

@svalat
Copy link
Member

svalat commented Mar 11, 2020

Thanks for the feedback. I will give a look tomorrow, I setted up a fedora 9 VM so I can look in the same conditions than you.

For the source:

  • have you tried the backtrace or libunwind mode for the stack:mode option. Enter-exit is there more for performance issue with codes making million allocations and use assembly code which can be wrong.
  • Have you compiled your program in debug mode (-g) ?

@Teraslilja
Copy link
Author

I tried different combinations of libunwind on | off & backtrace | enter-exit, but nothing changes. That was my reason to ask, could it be possible that the configuration file config.ini is not actually read and processed.

Yes, the -g switch is used. The exact compiler and linker options/switches are:
C++: -O0 -g3 -finstrument-functions --coverage -isystem /opt/malt/include
Linker: -g3 --coverage /opt/malt/lib64/libmalt.so

An enchantment for consideration: Collect automatically (without e.g. LCOV lib) coverage information about all allocation/release functions, so that it is possible to detect, are all of them covered with test

@svalat
Copy link
Member

svalat commented Mar 15, 2020

I merged into master the fixes.

I'm not sure to understand your issue, I made a run and got the call stacks well extracted under Fedora 9 i386 in an VM.

  • Looking more on your text I understood you linked MALT to your application ? The common use case if just compiling your app with -g and using malt yourapp.
  • Can you provide the exact launch command you are using ?
  • Can you make a test in the malt build directory by running ./src/lib/tests/simple-case-finstr-linked and looking the output file.

@Teraslilja
Copy link
Author

I did check the latest commit, and there were much more changes that I did (I changed just the type of axis::max to uint64_t, no other changes). Perhaps that explains ...

@Teraslilja
Copy link
Author

After several tries, I have not managed to get the assert to fail again. I will inform, if I get the assert again. So most likely the extensive type size_t changes to uint64_t solved that one.

@Teraslilja
Copy link
Author

Teraslilja commented Mar 16, 2020

The stack issue is more interesting one. I have launched the app with two different ways:

  1. Included the Malt lib during linking phase and
  2. The 'traditional way' - /opt/malt/bin/malt -d -v -c ./config.ini ../pc_malt/src/app-name (as root - the app requires)

The Malt's verbose logging does not indicate any issues related to the missing stack data.

The case 2)'s finally generated configuration is identical with supposed/included configuration at config.ini. The content of configuration is:

[time]
enabled = true
points = 1000
linear-index = false

[stack]
enabled = true
mode = enter-exit
resolve = true
libunwind = true
skip = 4

[output]
name = malt-%1-%2.%3
lua = false
json = true
callgrind = false
indent = false
config = true
verbosity = verbose
stack-tree = false
loop-suppress = false

[max-stack]
enabled = true

[distr]
alloc-size = true
realloc-jump = true

[trace]
enabled = false

[info]
hidden = false

[filter]
exe =
childs = true
enabled = true

[dump]
on-signal =
after-seconds = 0

@Teraslilja
Copy link
Author

Teraslilja commented Mar 16, 2020

I replaced the "-g3" with "-g" and no difference with stack info (stack report is empty)

@Teraslilja
Copy link
Author

I have noticed one change since the latest commit, the backtrace information has been missing or calculated incorrectly (both sources and calltree) reports
Screenshot from 2020-03-16 13-24-46
) :/
Screenshot from 2020-03-16 13-28-06

@Teraslilja
Copy link
Author

I run the test ./src/lib/tests/simple-case-finstr-linked, and it generated this JSON file
malt-simple-case-finstr-linked-28749.zip

@Teraslilja
Copy link
Author

Feedback from the "Global variables" report:
It would be ideal, if the binaries and variables that belong to Malt, are not included into report
Screenshot from 2020-03-16 13-45-34 OR they do not affect, what is included into "others" category

@Teraslilja
Copy link
Author

I forgot to mention; the malt-view is run at 64-bit side (host of Fedora). Could it affect the results some how?

@Teraslilja
Copy link
Author

This is hopefully the last 32/64 -bitness issue:

$ git diff
diff --git a/src/lib/tools/ProcPagemapReader.hpp b/src/lib/tools/ProcPagemapReader.hpp
index b64e7dc1..4c9505c5 100644
--- a/src/lib/tools/ProcPagemapReader.hpp
+++ b/src/lib/tools/ProcPagemapReader.hpp
@@ -23,9 +23,9 @@ namespace MALT
//Infos from https://www.kernel.org/doc/Documentation/vm/pagemap.txt
struct ProcPageMapEntry
{
- unsigned long pfn:55; // Bits 0-54 page frame number (PFN) if present
- // Bits 0-4 swap type if swapped
- // Bits 5-54 swap offset if swapped
+ uint64_t pfn:55; // Bits 0-54 page frame number (PFN) if present
+ // Bits 0-4 swap type if swapped
+ // Bits 5-54 swap offset if swapped
unsigned char dirty:1; // Bit 55 pte is soft-dirty (see Documentation/vm/soft-dirty.txt)
unsigned char zero:5; // Bits 56-60 zero
unsigned char file:1; // Bit 61 page is file-page or shared-anon

@Teraslilja
Copy link
Author

I have tested and verified that "-fintrument-functions" is 'implemented' and callback EnterExitStack::enterFunction() is called.

But for some reason, the variable "size" is either [at first] two (2) for a long duration and the later three (3) at the end of callback before returning. This doesn't sound correct at all.

@Teraslilja
Copy link
Author

Teraslilja commented Mar 17, 2020

The problem most likely is related to instrumentation, because, when I switched to "backtrace" with libunwind, then I managed to get call trees, but stack report is still empty (requires working instrumentation?)

@svalat
Copy link
Member

svalat commented Mar 21, 2020

Hi,
the stack report require the -finstrument-function, but from what you are saying it might be buggy on x86, I need to check.

The ProcPageMapEntry class is not yet used by malt, this is for a future feature, I will give a look as the struct might be different on i386.

There is no issue to run the webview on another machine and there is no 32/64 bit issue as all is javascript for the view.

Except for the stack size tracking, did you got what you wanted in backtrace/libunwind mode ?

Thanks for all the feedback.

I will give a look for the enter-exit.

@Teraslilja
Copy link
Author

I did have to update the Fedora 9's libunwind, it crashed during a long run. I did install version 1.2.1 from sources and no more crashes. Othwerwise backtrace&libunwind is now working like a charm.

@Teraslilja
Copy link
Author

I checked the JSON files generated and the stack parts basically contained only 0x1, which is by default added by Malt code during "enter-exit" mode (as indication of end of list?). It looks like that the enter-exit callbacks are working & called, but the entered function pointer is not added to the list

@Teraslilja
Copy link
Author

One curious case with webview: A function "FT_Alloc" leaks (part of freetype lib)

  • Sources can tell even the location of function: "/usr/src/pc_libs/freetype-2.1.3/src/base/ftutil.c | FT_Alloc", while ..
  • Calltree claims "Could not find the selected function."

The debuginfo package is installed for freetype lib. What could be reason for this?

@Teraslilja
Copy link
Author

Teraslilja commented Mar 24, 2020

I have tested more the function instrumentation and noticed that the enterFunction() were called with funcAddr == 0x1. I also noticed that the enterFunction were NOT always called, when expected.

Further more, if the instrumentation is done this order "--coverage -finstrument-functions", NO coverage files (.gcno) were created.

So it looks like coverage and function instrumentation together leads to problems. But without coverage functionality, there were no change with function instrumentation

@Teraslilja
Copy link
Author

Apparently I have checked only the alloc/release callbacks for stack. The function __cyg_profile_func_enter() were not called a single time

@svalat
Copy link
Member

svalat commented Mar 28, 2020

I'm not sure to follow your bug with __cyg_profile_func_enter(), I made a test and it is working.

What is the exact compile command you are using ?

svalat added a commit that referenced this issue Mar 28, 2020
@svalat svalat modified the milestones: v1.2.0, v1.3.0 Mar 28, 2020
@Teraslilja
Copy link
Author

I'm not sure to follow your bug with __cyg_profile_func_enter(), I made a test and it is working.

What is the exact compile command you are using ?

I will answer this, when I am back at work laptop. But the building is done with Fedora 9's kDevelop.

FYI: I used the attached pic to demonstrate the Malt's excellency to detect memory usage. With Source and Calltree tabs, I showed to the my client the places that have allocated the most memory during peak usage. And with leak information showed only several hundred kilos of memory in the same time. They were really impressed.

While doing this I noticed two issues or suggestions for improvement:

  1. When at excessive allocated memory function at Source tab, I want backtrace the call tree (to find a place to add e.g. release memory or remove pointer). Opening the calltree at root starting from main() is frustrating. Better would be, if I could start from the selected function, in this case the function with excessive amount of allocated memory, and approach toward main()

  2. In some cases the calltree tab failed to show sources of selected function (to separate browser tab), even though there were source available for the function. I will give more details, when back at work. I suspect that this is related to case, where there are more than one calling instances from the parent function

Screenshot from 2020-03-27 10-59-46

@Teraslilja
Copy link
Author

Teraslilja commented Mar 30, 2020

The makefile is autogenerated by kDevelop and run by kDevelop.

The compiler version is: g++ (GCC) 4.3.0 20080428 (Red Hat 4.3.0-8)

C++ source is compiled as

$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT <source>.o -MD -MP -MF $(DEPDIR)/<source>.Tpo -c -o <source>.o `test -f '<source>.cpp'

while the linking happens as

$(CXXLD) $(AM_CXXFLAGS) $(CXXFLAGS) $(AM_LDFLAGS) $(LDFLAGS) -o $@ $(&lt;target&gt;_LDADD) $(LIBS)

Now,
$(DEFS) has a single define
$(DEFAULT_INCLUDES) and $(INCLUDES) have several include paths
$(AM_CXXFLAGS) seems to be empty/undefined
$(CPPFLAGS) contains only definitions
$(AM_CXXFLAGS) seems to be empty/undefined
$(CXXFLAGS) has "-O0 -g3 -B -finstrument-functions --coverage -ffast-math -fno-rtti -fno-strict-aliasing" with warnings "-Wall -Wno-switch -Wno-format" and an include path
$(AM_LDFLAGS) seems to be empty/undefined
$(LDFLAGS) has "/opt/malt/lib/libmalt.so -finstrument-functions --coverage " and a linked gui lib
$(<target>_LDADD) has several static libs, include path for a single lib search path and several shared libs
$(LIBS) is empty

@Teraslilja
Copy link
Author

One curious case with webview: A function "FT_Alloc" leaks (part of freetype lib)

* Sources can tell even the location of function: "/usr/src/pc_libs/freetype-2.1.3/src/base/ftutil.c | FT_Alloc", while ..

* Calltree claims "Could not find the selected function."

The debuginfo package is installed for freetype lib. What could be reason for this?

I did write a test application to see, does two (or more) calls at parent function affect this. No. I also added option "-fno-inline" for compilation and it either didn't solve the issue.

@Teraslilja
Copy link
Author

Background:
I have used Malt with code that is very bad C++ code, which looks like a Java and C coders are implemented based on C++98 standard. Basically this means that classes have init() function (often without constructors), close() function (often without destructor) and too often resources of objects are released outside of classes (not at destructor). So as no surprise, there are plenty of std::vector<> leaks as the data is typically hierarchically ordered as class1::vector<class2::vector<class3::vector>>. And the inner most leaking vector has been used in dozens of different "structures" of data. A pure nightmare to figure out the leaking place O_o

Suggestion for improvement:
The Malt does have two views to see e.g. leaks Source tab to see information per source line and Calltree tab to see information through call hierarchy. With tracking, would it be possible to see the information like these two tab, but for the data structures and/or variables?

@svalat
Copy link
Member

svalat commented Oct 17, 2020

Sorry I let it down for a long time. Aswers to your questions :

  • No there is no "simple" way to map the allocations on the variables even if your are right this can be in interesting info.
  • About --coverage and --finstrument-function I think the coverage utils exploit the --finstrument-function so you might not enable coverage and use malt finstrument mode at the same time.

Can I close the issue ?

@Teraslilja
Copy link
Author

Yes

@svalat svalat closed this as completed Oct 17, 2020
@svalat svalat unpinned this issue Oct 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants