Skip to content

Conversation

@Jongy
Copy link
Contributor

@Jongy Jongy commented Jul 12, 2022

fdtransfer works by fork()ing, returning control to its parent while the child process waits until
async-profiler connects. The child inherits the standard stream fds, preventing subprocess's communicate
from returning when the process exits. I added close() calls and now it works.

The rationale - I'm seeing errors in _run_fdtransfer but I'm unable to debug because all I get is:

Traceback (most recent call last):
  File "/app/gprofiler/profilers/profiler_base.py", line 139, in snapshot
    result = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/gprofiler/profilers/java.py", line 953, in _profile_process
    stackcollapse = self._profile_ap_process(ap_proc, comm)
  File "/app/gprofiler/profilers/java.py", line 958, in _profile_ap_process
    started = ap_proc.start_async_profiler(self._interval, ap_timeout=self._ap_timeout)
  File "/app/gprofiler/profilers/java.py", line 519, in start_async_profiler
    self._run_fdtransfer()
  File "/app/gprofiler/profilers/java.py", line 506, in _run_fdtransfer
    run_process(
  File "/app/gprofiler/utils/__init__.py", line 265, in run_process
    raise CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
gprofiler.exceptions.CalledProcessError: Command '['/app/gprofiler/resources/java/fdtransfer', '54327']' returned non-zero exit status 1. 
stdout: None
stderr: None

with this PR, and this small patch on fdtransfer:

diff --git a/src/fdtransfer/fdtransferServer.cpp b/src/fdtransfer/fdtransferServer.cpp
index fad31b6..bf5a7b5 100644
--- a/src/fdtransfer/fdtransferServer.cpp
+++ b/src/fdtransfer/fdtransferServer.cpp
@@ -53,7 +53,7 @@ int FdTransferServer::_peer;
 
 bool FdTransferServer::bindServer(struct sockaddr_un *sun, socklen_t addrlen, int accept_timeout) {
     _server = socket(AF_UNIX, SOCK_SEQPACKET, 0);
-    if (_server == -1) {
+    if (_server == -1 || 1) {
         perror("FdTransfer socket()");
         return false;

I'm now getting:

Traceback (most recent call last):
  File "/app/gprofiler/profilers/profiler_base.py", line 119, in _wait_for_profiles
    result = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/gprofiler/profilers/java.py", line 953, in _profile_process
    stackcollapse = self._profile_ap_process(ap_proc, comm, duration)
  File "/app/gprofiler/profilers/java.py", line 958, in _profile_ap_process
    started = ap_proc.start_async_profiler(self._interval, ap_timeout=self._ap_timeout)
  File "/app/gprofiler/profilers/java.py", line 518, in start_async_profiler
    self._run_fdtransfer()
  File "/app/gprofiler/profilers/java.py", line 506, in _run_fdtransfer
    run_process(
  File "/app/gprofiler/utils/__init__.py", line 265, in run_process
    raise CalledProcessError(retcode, process.args, output=stdout, stderr=stderr)
gprofiler.exceptions.CalledProcessError: Command '['/app/gprofiler/resources/java/fdtransfer', '2580351']' returned non-zero exit status 1. 
stdout: b''
stderr: b'FdTransfer socket(): Success\n'

so I will be able to debug real problems later on.

See commit Granulate/async-profiler@83fd518.

fdtransfer works by fork()ing, returning control to its parent while the child process waits until
async-profiler connects. The child inherits the standard stream fds, preventing subprocess's communicate
from returning when the process exits. I added close() calls and now it works.
@Jongy Jongy added the enhancement New feature or request label Jul 12, 2022
@Jongy Jongy requested a review from michelhe July 12, 2022 22:25
@Jongy Jongy mentioned this pull request Jul 14, 2022
@Jongy Jongy merged commit 58429ea into master Jul 14, 2022
@Jongy Jongy deleted the java-fdtransfer-output branch July 14, 2022 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants