New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FreeBSD buildbots cannot compile Python #90421
Comments
Seems that the FreeBSD buildbots cannot compile Python 3.11 anymore: LD_LIBRARY_PATH=/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build CC='cc -pthread' LDSHARED='cc -pthread -shared ' OPT='-g -O0 -Wall' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build
Traceback (most recent call last):
File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/./setup.py", line 49, in <module>
from distutils.command.build_ext import build_ext
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/command/build_ext.py", line 13, in <module>
from distutils.sysconfig import customize_compiler, get_python_version
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/distutils/sysconfig.py", line 53, in <module>
_config_vars = get_config_vars()
^^^^^^^^^^^^^^^^^
File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 621, in get_config_vars
_init_posix(_CONFIG_VARS)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/sysconfig.py", line 482, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '_sysconfigdata_d_freebsd14_x86_64-unknown-freebsd14'
*** [sharedmods] Error code 1
make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build
make: stopped in /usr/home/buildbot/python/3.x.koobs-freebsd-564d/build |
There is also this other builder: https://buildbot.python.org/all/#/builders/172/builds/1093 failing what what seems is a freeze problem: - 'use_frozen_modules': 1,
+ 'use_frozen_modules': -1,
? +
'use_hash_seed': 0,
'user_site_directory': 1,
'verbose': 0,
'warn_default_encoding': 0,
'warnoptions': [],
'write_bytecode': 1,
'xoptions': []} Ran 1 test in 0.152s Adding Eric and Guido for this particular problem to the nosy list |
Not to point out the obvious but this buildbot uses --enable-shared which is often a source of build / bootstrap issues, such as the compiler executable being built ends up dynamically loading a libpython from a previous installation or build. |
On my FreeBSD 13 VM the sysconfig data module is build/lib.freebsd-13.0-RELEASE-amd64-3.11-pydebug/sysconfigdata_d_freebsd13.py It does not have any multiarch suffix. The multiarch suffix is detected by configure.ac. Did FreeBSD 14.0's CC get support for "cc --print-multiarch"? I found a related FreeBSD bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258377 |
I have tested my PR on a FreeBSD VM. Python fails to compile on main and builds correctly with my patch. |
I have backported the patch to all versions that receive regular fixes. |
I closed the bug without intention to do so. Sorry for the noise. |
FTR:
|
The test_capi check for check_pyobject_freed_is_freed() is failing because FreeBSD 14.0 uses jemalloc, which performs its own dead memory cleaning. jemalloc's free() fills freed memory with byte 0x5a, which overrides Python's 0xdd marker. $ PYTHONMALLOC=malloc_debug gdb ./python
(gdb) b check_pyobject_freed_is_freed
(gdb) b _PyMem_DebugRawFree
(gdb) disable 2
(gdb) run -X faulthandler t.py
...
Breakpoint 1, check_pyobject_freed_is_freed (self=0x800f16fe0, _unused_args=0x0) at Modules/_testcapimodule.c:5032
5032 PyObject *op = PyObject_CallNoArgs((PyObject *)&PyBaseObject_Type);
(gdb) enable 2
(gdb) c
Continuing. Breakpoint 2, _PyMem_DebugRawFree (ctx=0x5f8ad8 <_PyMem_Debug+96>, p=0x800e1a0a0) at Objects/obmalloc.c:2544 |
$ PYTHONMALLOC=malloc_debug ./python t.py
Traceback (most recent call last):
File "/usr/home/heimes/dev/python/cpython/t.py", line 4, in <module>
_testcapi.check_pyobject_freed_is_freed()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_testcapi.error: check_pyobject_freed_is_freed: object is not seen as freed
$ echo $?
1 $ MALLOC_CONF="junk:false" PYTHONMALLOC=malloc_debug ./python t.py
$ echo $?
0 |
Is this under control, or do you still need help with some part? (Are there |
test_capi is fixed now. test_embed is still an issue. |
In summary, regarding the test_embed failure:
...
...
Note that in-tree builds are working find on other platforms, AFAICS. |
As Christian mentions, we still have the test_embed issue |
Yes, almost all of the buildbots are using in-tree builds. IIRC, buildbots for out-of-tree builds were added just recently. |
AFAICS, the test_embed failure first appeared after #73227 (bpo-45582) was merged. After #73227, there has been numerous fixes for test_embed: $ git log --oneline | grep bpo-45582
3f398a77d3 bpo-45582: Fix test_embed failure during a PGO build on Windows (GH-30014)
b0b3086279 bpo-45582: Write empty pybuilddir.txt on Windows to allow relocatable build directories (GH-29979)
06c4ae8b13 bpo-45582: Fix framework path and bootstrap build (GH-29954)
b7ef27bc08 bpo-45582: Ensure PYTHONHOME still overrides detected build prefixes (GH-29948)
f16f93e527 bpo-45582: framework build: modPath must not be const (GH-29944)
af1db4eb55 bpo-45582: Fix getpath_isxfile() and test_embed on Windows (GH-29930)
628abe4463 bpo-45582: Fix signature of _Py_Get_Getpath_CodeObject (GH-29921)
0ae4e0c959 bpo-45582 Fix prototype of _Py_Get_Getpath_CodeObject. (GH-29907)
7d7c91a8e8 bpo-45582: Add a NOT operator to the condition in getpath_isxfile (GH-29906)
ccb73a0d50 bpo-45582: Fix out-of-tree build issues with new getpath (GH-29902)
99fcf15052 bpo-45582: Port getpath[p].c to Python (GH-29041) |
Make that "first appeared on FreeBSD". Also, numerous was perhaps an exaggeration. Make that a few :) |
Just to be clear, the test_embed failure is on AMD64 FreeBSD Non-Debug 3.x (https://buildbot.python.org/all/#/builders/172). I agree that 99fcf15 (bpo-45582: Port getpath[p].c to Python (GH-29041)) is likely to be the cause. |
IIUC Steve is still on vacation, so we shouldn't wait for him to fix this. |
A FreeBSD builder where it's working: AMD64 FreeBSD Shared 3.x (koobs-freebsd-564d) Where it isn't: AMD64 FreeBSD Non-Debug 3.x (koobs-freebsd-9e36) The main difference I see is the FreeBSD version. |
Also, AMD64 FreeBSD Shared 3.x is configured --with-pydebug and --enable-shared. I'd be surprised if those were the root cause of this though. Both buildbots are in-tree builds. FWIW, my FreeBSD 14.0-CURRENT main-n250453-7ac82c96fe7 VM also fails, as noted in previous posts. It's built with just ./configure && make. |
I just put up a PR that should fix the failure. |
Great. I tried the same trick on my VM earlier today, but I was unsure if it was the correct fix :) Thanks! |
test_pyobject_freed_is_freed is still failing on AMD64 FreeBSD Shared 3.x. |
Both FreeBSD bots are green again. Thank you Christian and Eric! |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: