FreeBSD 12.1: setproctitle() has no effect

[jeroenp]

I just upgraded setproctitle from 1.1.10 to 1.2.1 and get the impression that it stopped working.

The changesets from issue #67 seems to set more macro defines than in the past. And src/spt_status.c does lots of things with macros... sooo, yes, maybe different things happen than in the previous situation. I find it hard to comprehend the various ifdef/define combinations.

FreeBSD 12 supports setproctitle(3) (in unistd.h).

$ python3.7 -m venv setproctitle                                                     
$ . setproctitle/bin/activate 
(setproctitle) $ pip install /tmp/setproctitle-1.2.1.tar.gz                          
Processing /tmp/setproctitle-1.2.1.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: setproctitle
  Building wheel for setproctitle (PEP 517) ... done
  Created wheel for setproctitle: filename=setproctitle-1.2.1-cp37-cp37m-freebsd_12_1_release_p10_amd64.whl size=11279 sha256=38c2ee5eaf085a9468c15b62c526428a22c2d752477585d92e648edd8de7ed8a
  Stored in directory: /tmp/pip-ephem-wheel-cache-joal_phl/wheels/1e/8a/fb/020e579b6877f2c3184f752af1272a56901da406f5fc051c3b
Successfully built setproctitle
Installing collected packages: setproctitle
Successfully installed setproctitle-1.2.1

(setproctitle) $ python
Python 3.7.9 (default, Aug 27 2020, 07:44:13) 
[Clang 8.0.1 (tags/RELEASE_801/final 366581)] on freebsd12
Type "help", "copyright", "credits" or "license" for more information.
>>> import setproctitle
>>> setproctitle.getproctitle()
''
>>> [1] + Suspended               python
(setproctitle) $ ps
  PID TT  STAT    TIME COMMAND
13106  2  SJ   0:00.01 /bin/sh -i
13893  2  TJ   0:00.02 python (python3.7)
13957  2  R+J  0:00.00 ps
(setproctitle) $ fg
python

>>> setproctitle.setproctitle("helloworld")
>>> setproctitle.getproctitle()
''
>>> [1] + Suspended               python
(setproctitle) $ ps
  PID TT  STAT    TIME COMMAND
13106  2  SJ   0:00.01 /bin/sh -i
13893  2  TJ   0:00.03 python (python3.7)
14016  2  R+J  0:00.00 ps

[dvarrazzo]

What happens if you try running your test python script using the SPT_DEBUG env var to 1?

What is you sys.platform?

[jeroenp]

See below, find_argv_from_env() bails out.

The sys.platform is freebsd12.

(setproctitle) $ export SPT_DEBUG=1
(setproctitle) $ python
Python 3.7.9 (default, Aug 27 2020, 07:44:13) 
[Clang 8.0.1 (tags/RELEASE_801/final 366581)] on freebsd12
Type "help", "copyright", "credits" or "license" for more information.
>>> import setproctitle
[SPT]: module init
>>> setproctitle.getproctitle()
[SPT]: reading argc/argv from Python main
[SPT]: found 1 arguments
[SPT]: walking from environ to look for the arguments
[SPT]: found environ at 0x800a301a0
[SPT]: argv[0] should be at 0x800a30199
[SPT]: argv[0] doesn't match 'python'
[SPT]: couldn't find argv from environ
[SPT]: get_argc_argv failed
''
>>> [1] + Suspended               python
(setproctitle) $ ps
  PID TT  STAT    TIME COMMAND
23001  1  SJ   0:00.01 /bin/sh -i
23388  1  TJ   0:00.03 python (python3.7)
23484  1  R+J  0:00.00 ps
(setproctitle) $ fg
python


>>> setproctitle.setproctitle("hello world")
[SPT]: failed to initialize setproctitle
>>> [1] + Suspended               python
(setproctitle) $ ps
  PID TT  STAT    TIME COMMAND
23001  1  SJ   0:00.01 /bin/sh -i
23388  1  TJ   0:00.03 python (python3.7)
23556  1  R+J  0:00.00 ps
(setproctitle) $ fg
python

>>> import sys
>>> sys.platform
'freebsd12'
>>>
(setproctitle) $ uname -a
FreeBSD hostname 12.1-RELEASE-p10 FreeBSD 12.1-RELEASE-p10 GENERIC  amd64
(setproctitle) $ python3.7 --version
Python 3.7.9

[dvarrazzo]

[SPT]: argv[0] doesn't match 'python'

That's weird, isn't it? And the distance between arg[0] and env is 7 bytes, which is consistent with "python\x00"

why does this strcmp doensn't return 0?

    if (strcmp(ptr, arg0)) {
        spt_debug("argv[0] doesn't match '%s'", arg0);
        goto exit;
    }

could you please try to apply this patch:

diff --git a/src/spt_setup.c b/src/spt_setup.c
index 8953786..17597f7 100644
--- a/src/spt_setup.c
+++ b/src/spt_setup.c
@@ -213,7 +213,7 @@ find_argv_from_env(int argc, char *arg0)
         goto exit;
     }
     if (strcmp(ptr, arg0)) {
-        spt_debug("argv[0] doesn't match '%s'", arg0);
+        spt_debug("argv[0] '%s' doesn't match '%s'", ptr, arg0);
         goto exit;
     }
 

and tell what is the debug output?

Thank you!

[jeroenp]

ptr is an empty string:

[SPT]: argv[0] '' doesn't match 'python'

(meanwhile I'm trying to get my head around the reverse pointer scan; Let me know where I can add more debugging)

[dvarrazzo]

I don't know, the problems seem there around. Can you dump the numeric value of the chars between argv[0] and environ?

[jeroenp]

no issues here when argv > 1:

(setproctitle) $ python -c 'import setproctitle; setproctitle.setproctitle("he")'
[SPT]: module init
[SPT]: reading argc/argv from Python main
[SPT]: found 3 arguments
[SPT]: walking from environ to look for the arguments
[SPT]: found environ at 0x7fffffffed9f
[SPT]: found argv[2] at 0x7fffffffed6a: import setproctitle; setproctitle.setproctitle("he")
[SPT]: found argv[1] at 0x7fffffffed67: -c
[SPT]: argv[0] should be at 0x7fffffffed60
[SPT]: found argv[0]: python

[jeroenp]

I have next to no C skills, so here goes.. Looks like just NULL's.

--- /jails/data/edp-staging/tmp/setproctitle-1.2.1/src/spt_setup.c.orig 2020-12-18 18:23:45.612310000 +0100
+++ /jails/data/edp-staging/tmp/setproctitle-1.2.1/src/spt_setup.c      2020-12-18 20:38:44.522783000 +0100
@@ -182,6 +182,19 @@
         goto exit;
     }
     spt_debug("found environ at %p", ptr);
+ 
+    spt_debug(".first env is: %s", ptr);
+    {
+        char *pptr;
+        size_t j;
+        for (j = 0; j < strlen(arg0); j++) {
+            pptr = ptr - 1 - j;
+            spt_debug(".ptr %p val: %i", pptr, *pptr);
+        }
+        pptr--;
+        spt_debug(".ptr %p val: %i", pptr, *pptr);
+    }
+
     limit = ptr - ARG_MAX;
     --ptr;
     for (i = argc - 1; i >= 1; --i) {

>>> setproctitle.setproctitle("ho ho")
[SPT]: reading argc/argv from Python main
[SPT]: found 1 arguments
[SPT]: walking from environ to look for the arguments
[SPT]: found environ at 0x800a30180
[SPT]: .first env is: COLUMNS=117
[SPT]: .ptr 0x800a3017f val: 0
[SPT]: .ptr 0x800a3017e val: 0
[SPT]: .ptr 0x800a3017d val: 0
[SPT]: .ptr 0x800a3017c val: 0
[SPT]: .ptr 0x800a3017b val: 0
[SPT]: .ptr 0x800a3017a val: 0
[SPT]: .ptr 0x800a30179 val: 0
[SPT]: argv[0] should be at 0x800a30179
[SPT]: argv[0] '' doesn't match 'python'
[SPT]: couldn't find argv from environ
[SPT]: get_argc_argv failed
[SPT]: failed to initialize setproctitle

[dvarrazzo]

Would you be so kind to check what happened instead with setproctitle 1.1? Thank you very much

[jeroenp]

Sorry it took a while since my last comment. It's been a bit busy in my life.

Meanwhile the jail I where I was testing this issue has been upgraded from python3.7 to python3.9. I've repeated the earlier steps with the same results. And now also with 1.1.10: The previous version gives the exact same complaints as 1.2, but... it works:

(setproctitle1110) $ python
Python 3.9.1 (default, Jan 27 2021, 16:10:45) 
[Clang 10.0.1 (git@github.com:llvm/llvm-project.git llvmorg-10.0.1-0-gef32c611a on freebsd12
Type "help", "copyright", "credits" or "license" for more information.
>>> import setproctitle
[SPT]: module init
[SPT]: reading argc/argv from Python main
[SPT]: found 1 arguments
[SPT]: walking from environ to look for the arguments
[SPT]: found environ at 0x800a941b0
[SPT]: .first env is: COLUMNS=137
[SPT]: .ptr 0x800a941af val: 0
[SPT]: .ptr 0x800a941ae val: 0
[SPT]: .ptr 0x800a941ad val: 0
[SPT]: .ptr 0x800a941ac val: 0
[SPT]: .ptr 0x800a941ab val: 0
[SPT]: .ptr 0x800a941aa val: 0
[SPT]: .ptr 0x800a941a9 val: 0
[SPT]: argv[0] should be at 0x800a941a9
[SPT]: argv[0] '' doesn't match 'python'
[SPT]: couldn't find argv from environ
[SPT]: get_argc_argv failed
[SPT]: failed to initialize module setproctitle
>>> setproctitle.setproctitle("helloworld")


2008 58919  0.0  0.0  12336  3392  2  IJ   22:03    0:00.04 /bin/sh -i
2008 65716  0.0  0.0  19400  9812  2  S+J  22:33    0:00.03 - python: helloworld (python3.9)

[ds-cbo]

Some more observations:

1. When running in the Python REPL, environ[0] starts at the last environment variable, instead of the first (as is when running python non-interactively). Since ARG_MAX is very large, this doesn't really matter.

2. spt_setup is called for every spt_getproctitle or spt_setproctitle call, but only set handles its errors

3. Running

python -c 'import setproctitle; setproctitle.setproctitle("helloworld"); import time; time.sleep(2)' &
sleep 1
ps -x -o pid,command

yields
95308 python: helloworld (python3.8)

_clean_up_title indeed removes the (python3.8) since the procname is python3.8 (after the symlink), but it doesn't strip the python: at the start, which fails tests/setproctitle_test.py#test_weird_path.

Fixes

1. This is an issue outside the scope of this project, which doesn't need to be fixed for it to work.

2. If the spt_setup check in spt_setproctitle is removed (it is still called, its result is just ignored), the latest version works fine and as expected. This change was introduced in 485a6a5 but I can't quite see the reason why.

3. Replace dir / "python" with dir / os.path.basename(sys.executable) in test_weird_path.

I will soon make a PR with these fixes.

[dvarrazzo]

Closed by #99, to be released in 1.3.0.