Update Modules/Setup #89711
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
assignee = 'https://github.com/brettcannon' closed_at = <Date 2021-11-23.10:58:45.780> created_at = <Date 2021-10-20.23:58:48.199> labels = ['build', '3.11'] title = 'Update Modules/Setup' updated_at = <Date 2021-11-23.10:58:45.779> user = 'https://github.com/brettcannon'
activity = <Date 2021-11-23.10:58:45.779> actor = 'christian.heimes' assignee = 'brett.cannon' closed = True closed_date = <Date 2021-11-23.10:58:45.780> closer = 'christian.heimes' components = ['Build'] creation = <Date 2021-10-20.23:58:48.199> creator = 'brett.cannon' dependencies =  files =  hgrepos =  issue_num = 45548 keywords = ['patch'] message_count = 31.0 messages = ['404548', '404559', '404560', '404627', '404710', '404790', '404864', '404897', '404898', '404899', '404900', '404906', '404924', '404953', '404980', '404987', '404992', '405032', '405045', '405046', '405047', '405053', '405055', '405056', '405092', '405123', '405172', '405227', '405314', '406812', '406834'] nosy_count = 7.0 nosy_names = ['lemburg', 'brett.cannon', 'christian.heimes', 'ned.deily', 'zach.ware', 'Anthony Sottile', 'miss-islington'] pr_nums = ['29115', '29143', '29177', '29179', '29181', '29188', '29199', '29206', '29216', '29225', '29263', '29268'] priority = 'low' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = None url = 'https://bugs.python.org/issue45548' versions = ['Python 3.11']
The text was updated successfully, but these errors were encountered:
When trying to compile a completely static CPython interpreter I ran into the following issues:
And the following modules are completely missing from the file:
Status is now:
And the following modules are completely missing from the file:
I started https://github.com/python/cpython/compare/main...tiran:configure_pkgconfig?expand=1 to integrate pkg-config with configure. The patchset also contains examples how to pass flags from configure into Makefile and Modules/Setup.
I ran into another problem. In shared mode, Modules/Setup places the shared modules in ./Modules/ instead of the build directory ./build/lib.something/. The ./Modules/ directory is not on sys.path. getpath.c only adds the build dir to sys.path. It takes the value from ./pybuilddir.txt. The file is generated by sysconfig.py.
$ make ... *** WARNING: renaming "_asyncio" since importing it failed: No module named 'math'
The following modules found by detect_modules() in setup.py, have been
$ find -name 'math*.so' ./Modules/math.cpython-311-x86_64-linux-gnu.so
FYI: I've been working with a fixed Setup file in PyRun for a long while. There are indeed a number of modules missing from Setup, since the whole logic was left behind a bit after things moved to setup.py.
The issue with _math.o is actually in the main Makefile.pre.in. The version listed there does not match the Makefile lines added via Setup. For PyRun, I had to comment out the one in Makefile.pre.in and then only add one instance to the math module and not the cmath one. This avoids a (harmless) warning during the build.
I'm not sure what the _math.o entry exists in Makefile.pre.in. It's only needed by those two modules, AFAIK.
Here's the list of modules I had to add in the past (taken from the 3.10 port):
# _decimal needs more complex setup, punting on this for now
# _ctypes needs to build libffi first - punting on this
#_lzma _lzmamodule.c -llzma
# Optional add-on for multiprocessing to use shared memory #POSIXSHMEM_LIBS=rt POSIXSHMEM_LIBS= _posixshmem \ _multiprocessing/posixshmem.c \ -I$(srcdir)/Modules/_multiprocessing \ $(POSIXSHMEM_LIBS)
Not all modules are included, since I did not need all missing ones.
PR #73350 adds the missing modules and also introduces pkg-config lookups for dependencies.
Brett and I discussed that we have to introduce conditionals to Modules/Setup to make use of the AM_CONDITIONALs from PR #73350. We also have to figure out how to deal with system-libmpdec and system-cffi.
I'm using a very simple conditional logic in Setup, which is based
# Install the custom Modules/Setup file if test "$(MACOSX_PLATFORM)"; then \ sed -e 's/# @if macosx: *//' \ $(PYRUNDIR)/$(MODULESSETUP) > $(PYTHONDIR)/Modules/Setup; \ elif test "$(FREEBSD_PLATFORM)"; then \ sed -e 's/# @if freebsd: *//' \ $(PYRUNDIR)/$(MODULESSETUP) > $(PYTHONDIR)/Modules/Setup; \ else \ sed -e 's/# @if not macosx: *//' \ -e 's/# @if not freebsd: *//' \ $(PYRUNDIR)/$(MODULESSETUP) > $(PYTHONDIR)/Modules/Setup; \ fi;
Setup used to be templated as well in the past, but that logic was removed
On 23.10.2021 20:04, Christian Heimes wrote:
I think those patches are both taking things a bit too far.
This is a build problem, not a code problem. It's perfectly
The catch is that the makesetup logic is not smart enough to only
The entry in Makefile.pre.in should not be needed, since the logic
BTW: There's a simple trick to avoid the makesetup issue: simply
On 23.10.2021 21:30, Christian Heimes wrote:
I don't follow you. With the _math.o target in Makefile.pre.in,
And yes, it does export a symbol, but _Py_log1p is not going to
The trick is essentially not changing the 3.10 status quo. It
And it removes the warning of having multiple _math.o targets
OTOH, _math.h and .c are really small, so perhaps it's better
Today, only the _Py_log1p() code is actually used and then only
Brett, we can use AM_CONDITIONAL() to conditionally enable/disable a feature and AC_CONFIG_FILES() to create a Modules/Setup from a template:
AM_CONDITIONAL([HAVE_SCPROXY], [test "$ac_sys_system" = "Darwin"])
sets HAVE_SCPROXY_FALSE and HAVE_NIS_SCPROXY based on the check.
We can either do something like:
On 26.10.2021 16:17, Christian Heimes wrote:
I think it would be more helpful (and generate a lot fewer such
Some other things:
I saw that you stripped off lots of -I and -L options from the Setup
You also removed the structure of the listings, which makes things
Since Setup is a (potentially) hand edited file, it's good to leave
On 26.10.2021 19:42, Christian Heimes wrote:
Could Brett or you please add those notes back ? There's no other place
Regarding the -I and -L flags, my question was whether these are
I've gone through the Makefile and found that these are already
Please be careful when moving e.g. -I or -L options which point to
Having the -D defined in the relevant code is a lot better. Thanks
Editing Setup.local instead of Setup is not documented anywhere ?!
On 26.10.2021 20:47, Marc-Andre Lemburg wrote:
I could also edit the file and add those back, after you're done
I'm seeing some weird breakage in the deadsnakes builds, presumably due to this change:
It really depends on what "details" you're referring to. Most of what I removed were things like "Module by <person>", or saying _json.c is for "json accelerator" which is obvious to me. Anything that seemed pertinent to compilation I left in.
So if there's something you specifically want to add back in that you think is important then please feel free to as I'm done editing the file for my purposes at the moment.
On 27.10.2021 22:58, Brett Cannon wrote:
I had already listed some of those details.