Skip to content

Conversation

@Jongy
Copy link
Contributor

@Jongy Jongy commented Jul 12, 2021

Description

Build async-profiler with -static-libstdc++ to avoid the libstd++ dependency.

Needs #137

Related Issue

Possibly resolves #136

Motivation and Context

Allow async-profiler to be loaded also in environments without libstdc++.so.

How Has This Been Tested?

Checklist:

  • I have read the CONTRIBUTING document.
  • I have updated the relevant documentation.
  • I have added tests for new logic.

Jongy added 2 commits July 12, 2021 23:23
No reason to build on centos:6 really - centos:7 builds work on centos:6 as well.
@Jongy Jongy added the enhancement New feature or request label Jul 12, 2021
@Jongy Jongy requested a review from liorgorb July 12, 2021 20:28
And anyway there's no reason not to build statically.
liorgorb
liorgorb previously approved these changes Jul 12, 2021
liorgorb
liorgorb previously approved these changes Jul 13, 2021
@Jongy
Copy link
Contributor Author

Jongy commented Jul 13, 2021

I installed 8.0.292.hs-adpt via sdkman, (sdk install java 8.0.292.hs-adpt). It has no dependency on libstdc++.so. So I ran a Java app with it, and removed /usr/lib/x86_64-linux-gnu/libstdc++.so.6.

Result before (on master):

[2021-07-13 11:27:41,786] DEBUG: gprofiler.utils: Running command: (/app/gprofiler/resources/java/jattach 935723 load /tmp/gprofiler_tmp/async-profiler-v2.0g3/libasyncProfiler.so true start,event=itimer,file=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.output,collapsed,ann,sig,interval=90909090,framebuf=2000000,log=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.log)
[2021-07-13 11:27:41,809] DEBUG: gprofiler.utils: (['/app/gprofiler/resources/java/jattach', '935723', 'load', '/tmp/gprofiler_tmp/async-profiler-v2.0g3/libasyncProfiler.so', 'true', 'start,event=itimer,file=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.output,collapsed,ann,sig,interval=90909090,framebuf=2000000,log=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.log']) exit code: 255
[2021-07-13 11:27:41,809] DEBUG: gprofiler.utils: (['/app/gprofiler/resources/java/jattach', '935723', 'load', '/tmp/gprofiler_tmp/async-profiler-v2.0g3/libasyncProfiler.so', 'true', 'start,event=itimer,file=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.output,collapsed,ann,sig,interval=90909090,framebuf=2000000,log=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.log']) stdout: b'Connected to remote JVM\nJVM response code = -1\n\n'
[2021-07-13 11:27:41,809] WARNING: gprofiler.java: async-profiler DSO was not loaded into 935723
[2021-07-13 11:27:41,810] ERROR: gprofiler.profiler_base: JavaProfiler: failed to profile process 935723
Traceback (most recent call last):
  File "/app/gprofiler/profiler_base.py", line 111, in snapshot
    result = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/gprofiler/java.py", line 306, in _profile_process
    return self._profile_ap_process(ap_proc)
  File "/app/gprofiler/java.py", line 309, in _profile_ap_process
    started = ap_proc.start_async_profiler(self._interval)
  File "/app/gprofiler/java.py", line 212, in start_async_profiler
    self._run_async_profiler(start_cmd)
  File "/app/gprofiler/java.py", line 204, in _run_async_profiler
    raise JattachException(e.returncode, e.cmd, e.stdout, e.stderr, self.process.pid, ap_log) from None
gprofiler.java.JattachException: Command '['/app/gprofiler/resources/java/jattach', '935723', 'load', '/tmp/gprofiler_tmp/async-profiler-v2.0g3/libasyncProfiler.so', 'true', 'start,event=itimer,file=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.output,collapsed,ann,sig,interval=90909090,framebuf=2000000,log=/tmp/gprofiler_tmp/tmpt8_s95r_/async-profiler-935723.log']' returned non-zero exit status 255. 
stdout: b'Connected to remote JVM\nJVM response code = -1\n\n'
stderr: b''
Java PID: 935723
async-profiler log:
(empty)

and on this branch, it works fine :)

@Jongy
Copy link
Contributor Author

Jongy commented Jul 13, 2021

Bleh, doesn't work on a real CentOS 6:

$ ldd /tmp/gprofiler_tmp/async-profiler-v2.0g4/libasyncProfiler.so 
/tmp/gprofiler_tmp/async-profiler-v2.0g4/libasyncProfiler.so: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /tmp/gprofiler_tmp/async-profiler-v2.0g4/libasyncProfiler.so)
	linux-vdso.so.1 =>  (0x00007ffe337e1000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fa79a5b1000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fa79a394000)
	librt.so.1 => /lib64/librt.so.1 (0x00007fa79a18b000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fa799f07000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa799cf1000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa79995c000)
	/lib64/ld-linux-x86-64.so.2 (0x0000556bd9219000)
$ cat /etc/centos-release
CentOS release 6.10 (Final)

Jongy added 2 commits July 13, 2021 15:31
This reverts commit 8b2aa02.

The assumption that it's okay was incorrect.
Need newer GCC for -static-libstdc++
@Jongy
Copy link
Contributor Author

Jongy commented Jul 13, 2021

Okay, reverted back to CentOS 6 (and commented so next time we won't forget).

I install devtoolset 7 and build with it. Tested now on my CentOS 6 machine - works fine. Also, no libstdc++.so in ldd.

@Jongy Jongy requested a review from liorgorb July 13, 2021 12:55
@Jongy
Copy link
Contributor Author

Jongy commented Jul 13, 2021

Unrelated error..

[2021-07-13 13:31:19,608] ERROR: gprofiler.profiler_base: PySpyProfiler: failed to profile process 667
Traceback (most recent call last):
  File "gprofiler/profiler_base.py", line 111, in snapshot
  File "concurrent/futures/_base.py", line 425, in result
  File "concurrent/futures/_base.py", line 384, in __get_result
  File "concurrent/futures/thread.py", line 56, in run
  File "gprofiler/python.py", line 63, in _profile_process
  File "gprofiler/utils.py", line 146, in run_process
  File "subprocess.py", line 863, in communicate
  File "subprocess.py", line 1527, in _communicate
  File "selectors.py", line 351, in register
  File "selectors.py", line 237, in register
  File "selectors.py", line 224, in _fileobj_lookup
  File "selectors.py", line 39, in _fileobj_to_fd
ValueError: Invalid file object: <_io.BufferedReader name=18>

@Jongy
Copy link
Contributor Author

Jongy commented Jul 13, 2021

Also checked my Java running on java:latest - it has /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20 loaded.

@Jongy Jongy merged commit 54490a5 into master Jul 13, 2021
@Jongy Jongy deleted the async-profiler-static-libstdcpp branch July 13, 2021 13:39
Jongy added a commit that referenced this pull request Nov 30, 2022
This PR upgrades async-profiler to v2.9.

Post upgrading to v2.9, the force_symbols.cpp hack didn't work. This PR standarizes the build for CentOS 6 compatibility:

* Remove devtoolset-7 installation - this is a remnant from #139 which enabled -static-libstdc++, which is not supported on CentOS 6 GCC (see description in that PR). However, we've since switched to CentOS 7 (in #304) so it's no longer relevant - GCC on CentOS 7 supports it.
* Install compat-glibc and build against it with "-I /usr/lib/x86_64-redhat-linux6E/include -B /usr/lib/x86_64-redhat-linux6E/lib64/".
* Remove the hack of force linking memcpy.

Closes: #563
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error when running gProfiler - Failed to profile Java process

3 participants