subprocess.Popen can pass incomplete environment when posix_spawn is used

[kulikjak]

Bug report

Bug description:

Subprocess created with posix_spawn can have different environment when compared to a process created with stardard fork/exec.

The reason is that posix_spawn uses os.environ (when no env was given), which is not affected by os.putenv and os.unsetenv. This is different from execv, which uses process environment (available through environ).

This can be easily reproduced with test_putenv_unsetenv modified to call subprocess.run with close_fds=False (which makes it use the posix_spawn call):

import subprocess
import sys
import os

name = "PYTHONTESTVAR"
value = "testvalue"
code = f'import os; print(repr(os.environ.get({name!r})))'

os.putenv(name, value)
proc = subprocess.run([sys.executable, '-c', code], check=True,
                      stdout=subprocess.PIPE, text=True, close_fds=False)
assert proc.stdout.rstrip() == repr(value)

os.unsetenv(name)
proc = subprocess.run([sys.executable, '-c', code], check=True,
                      stdout=subprocess.PIPE, text=True, close_fds=False)
assert proc.stdout.rstrip() == repr(None)

I found it when I patched Python with #113118 (on Solaris, but this is pretty platform independent, I think).

CPython versions tested on:

3.9, 3.11

Operating systems tested on:

Other

Linked PRs

• gh-113119 fix environment handling in subprocess.Popen when posix_spawn is used #113120
• gh-113119: Fix the macOS framework installer build #113268

[gpshead]

reopened due to @ned-deily reporting on a parallel issue #113117 about this change:

This breaks macOS framework builds such as those used by the python.org installer:

$ ./configure --enable-framework && make -j
[...]
./Modules/posixmodule.c:7170:19: error: use of undeclared identifier 'environ'
        envlist = environ;
                  ^
./Modules/posixmodule.c:7238:31: error: use of undeclared identifier 'environ'
    if (envlist && envlist != environ) {
                              ^

Note the comments elsewhere in posixmodule.c such as at line 1582:

#elif defined(WITH_NEXT_FRAMEWORK) || (defined(__APPLE__) && defined(Py_ENABLE_SHARED))
    /* environ is not accessible as an extern in a shared object on OSX; use
       _NSGetEnviron to resolve it. The value changes if you add environment
       variables between calls to Py_Initialize, so don't cache the value. */
    e = *_NSGetEnviron();

[kulikjak]

Sorry for that, and thank you for the fix. Since CI was green, I didn't look into it much more and missed that.