gh-109566: Fix regrtest Python options for WASM/WASI

[vstinner]

WASM and WASI buildbots use multiple PYTHON environment variables such as PYTHONPATH and _PYTHON_HOSTRUNNER. Don't use -E if the --python=COMMAND option is used.

• Issue: Unify ways to run the Python test suite #109566

[vstinner]

!buildbot wasm

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 4cc9752 🤖

The command will test the builders whose names match following regular expression: wasm

The builders matched are:

• wasm32-emscripten node (pthreads) PR
• wasm32-wasi PR
• wasm32-emscripten browser (dynamic linking, no tests) PR
• wasm32-emscripten node (dynamic linking) PR

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Debian 3.x has failed when building commit 91fb8da.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/49/builds/6684) and take a look at the build logs.
4. Check if the failure is related to this commit (91fb8da) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/49/builds/6684

Failed tests:

• test.test_concurrent_futures.test_shutdown

Failed subtests:

• test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolForkserverProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_concurrent_futures/test_shutdown.py", line 49, in test_interpreter_shutdown
    self.assertFalse(err)
AssertionError: b'Exception in thread Thread-1:\nTraceback (most recent call last):\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/threading.py", line 1066, in _bootstrap_inner\n    self.run()\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/concurrent/futures/process.py", line 344, in run\n    self.add_call_item_to_queue()\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/concurrent/futures/process.py", line 399, in add_call_item_to_queue\n    self.call_queue.put(_CallItem(work_id,\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/multiprocessing/queues.py", line 94, in put\n    self._start_thread()\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/multiprocessing/queues.py", line 177, in _start_thread\n    self._thread.start()\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/threading.py", line 985, in start\n    _start_new_thread(self._bootstrap, ())\nRuntimeError: can\'t create new thread at interpreter shutdown\nTraceback (most recent call last):\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/multiprocessing/forkserver.py", line 274, in main\n    code = _serve_one(child_r, fds,\n           ^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/multiprocessing/forkserver.py", line 313, in _serve_one\n    code = spawn._main(child_r, parent_sentinel)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/multiprocessing/spawn.py", line 132, in _main\n    self = reduction.pickle.load(from_parent)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/multiprocessing/synchronize.py", line 115, in __setstate__\n    self._semlock = _multiprocessing.SemLock._rebuild(*state)\n                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n' is not false

[vstinner]

test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolForkserverProcessPoolShutdownTest.test_interpreter_shutdown

This is issue gh-109047.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 LTO + PGO 3.x has failed when building commit 91fb8da.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/568/builds/4924) and take a look at the build logs.
4. Check if the failure is related to this commit (91fb8da) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/568/builds/4924

Failed tests:

• test.test_multiprocessing_fork.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 22, done.        
remote: Counting objects:   4% (1/22)        
remote: Counting objects:   9% (2/22)        
remote: Counting objects:  13% (3/22)        
remote: Counting objects:  18% (4/22)        
remote: Counting objects:  22% (5/22)        
remote: Counting objects:  27% (6/22)        
remote: Counting objects:  31% (7/22)        
remote: Counting objects:  36% (8/22)        
remote: Counting objects:  40% (9/22)        
remote: Counting objects:  45% (10/22)        
remote: Counting objects:  50% (11/22)        
remote: Counting objects:  54% (12/22)        
remote: Counting objects:  59% (13/22)        
remote: Counting objects:  63% (14/22)        
remote: Counting objects:  68% (15/22)        
remote: Counting objects:  72% (16/22)        
remote: Counting objects:  77% (17/22)        
remote: Counting objects:  81% (18/22)        
remote: Counting objects:  86% (19/22)        
remote: Counting objects:  90% (20/22)        
remote: Counting objects:  95% (21/22)        
remote: Counting objects: 100% (22/22)        
remote: Counting objects: 100% (22/22), done.        
remote: Compressing objects:   9% (1/11)        
remote: Compressing objects:  18% (2/11)        
remote: Compressing objects:  27% (3/11)        
remote: Compressing objects:  36% (4/11)        
remote: Compressing objects:  45% (5/11)        
remote: Compressing objects:  54% (6/11)        
remote: Compressing objects:  63% (7/11)        
remote: Compressing objects:  72% (8/11)        
remote: Compressing objects:  81% (9/11)        
remote: Compressing objects:  90% (10/11)        
remote: Compressing objects: 100% (11/11)        
remote: Compressing objects: 100% (11/11), done.        
remote: Total 22 (delta 11), reused 16 (delta 11), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to '91fb8daa2494df4dd6a841ca8c742a03175c7ecd'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 91fb8daa24 gh-109566: Fix regrtest Python options for WASM/WASI (#109954)
Switched to and reset branch 'main'

find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
find: ‘build’: No such file or directory
make[2]: [Makefile:2821: clean-retain-profile] Error 1 (ignored)
./Include/internal/pycore_semaphore.h:57:18: warning: type of ‘_PySemaphore_Init’ does not match original declaration [-Wlto-type-mismatch]
 PyAPI_FUNC(void) _PySemaphore_Init(_PySemaphore *sema);
                  ^
Python/parking_lot.c:53:1: note: ‘_PySemaphore_Init’ was previously declared here
 _PySemaphore_Init(_PySemaphore *sema)
 ^
Python/parking_lot.c:53:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:49:1: warning: type of ‘_PySemaphore_Wait’ does not match original declaration [-Wlto-type-mismatch]
 _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout_ns, int detach);
 ^
Python/parking_lot.c:178:1: note: ‘_PySemaphore_Wait’ was previously declared here
 _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout, int detach)
 ^
Python/parking_lot.c:178:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:58:18: warning: type of ‘_PySemaphore_Destroy’ does not match original declaration [-Wlto-type-mismatch]
 PyAPI_FUNC(void) _PySemaphore_Destroy(_PySemaphore *sema);
                  ^
Python/parking_lot.c:81:1: note: ‘_PySemaphore_Destroy’ was previously declared here
 _PySemaphore_Destroy(_PySemaphore *sema)
 ^
Python/parking_lot.c:81:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:54:1: warning: type of ‘_PySemaphore_Wakeup’ does not match original declaration [-Wlto-type-mismatch]
 _PySemaphore_Wakeup(_PySemaphore *sema);
 ^
Python/parking_lot.c:197:1: note: ‘_PySemaphore_Wakeup’ was previously declared here
 _PySemaphore_Wakeup(_PySemaphore *sema)
 ^
Python/parking_lot.c:197:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Modules/_decimal/libmpdec/context.c:57: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time

./Modules/_decimal/libmpdec/context.c:57: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time

./Include/internal/pycore_semaphore.h:54:1: warning: type of ‘_PySemaphore_Wakeup’ does not match original declaration [-Wlto-type-mismatch]
 _PySemaphore_Wakeup(_PySemaphore *sema);
 ^
Python/parking_lot.c:197:1: note: ‘_PySemaphore_Wakeup’ was previously declared here
 _PySemaphore_Wakeup(_PySemaphore *sema)
 ^
Python/parking_lot.c:197:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:57:18: warning: type of ‘_PySemaphore_Init’ does not match original declaration [-Wlto-type-mismatch]
 PyAPI_FUNC(void) _PySemaphore_Init(_PySemaphore *sema);
                  ^
Python/parking_lot.c:53:1: note: ‘_PySemaphore_Init’ was previously declared here
 _PySemaphore_Init(_PySemaphore *sema)
 ^
Python/parking_lot.c:53:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:49:1: warning: type of ‘_PySemaphore_Wait’ does not match original declaration [-Wlto-type-mismatch]
 _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout_ns, int detach);
 ^
Python/parking_lot.c:178:1: note: ‘_PySemaphore_Wait’ was previously declared here
 _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout, int detach)
 ^
Python/parking_lot.c:178:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:58:18: warning: type of ‘_PySemaphore_Destroy’ does not match original declaration [-Wlto-type-mismatch]
 PyAPI_FUNC(void) _PySemaphore_Destroy(_PySemaphore *sema);
                  ^
Python/parking_lot.c:81:1: note: ‘_PySemaphore_Destroy’ was previously declared here
 _PySemaphore_Destroy(_PySemaphore *sema)
 ^
Python/parking_lot.c:81:1: note: code may be misoptimized unless -fno-strict-aliasing is used

make: *** [Makefile:2040: buildbottest] Error 5