bpo-43103: Add configure --without-static-libpython #24418

vstinner · 2021-02-02T18:12:53Z

Add configure --without-static-libpython option to not build
libpythonMAJOR.MINOR.a static library and not install python.o object
file.

Fix smelly.py and stable_abi.py tools when libpython3.10.a is
missing.

https://bugs.python.org/issue43103

vstinner · 2021-02-02T18:16:47Z

I tested 4 cases. CFLAGS="-O0" is just to build Python faster.

(1) disable shared without static

git clean -fdx
./configure CFLAGS="-O0" --without-static-python --prefix $PWD/target
make && make install

Output:

$ /bin/ls -1 target/lib/python3.10/config-3.10-x86_64-linux-gnu/
config.c
config.c.in
install-sh
Makefile
makesetup
__pycache__
python-config.py
Setup
Setup.local

$ find target/ -name "libpython*"
target/lib/libpython3.10.a

=> target/lib/libpython3.10.a is still installed. Should it be not installed?

(2) disable shared with static

git clean -fdx
./configure CFLAGS="-O0" --with-static-python --prefix $PWD/target
make && make install

Output:

$ /bin/ls -1 target/lib/python3.10/config-3.10-x86_64-linux-gnu/
config.c
config.c.in
install-sh
libpython3.10.a
Makefile
makesetup
__pycache__
python-config.py
python.o
Setup
Setup.local

$ find target/ -name "libpython*"
target/lib/python3.10/config-3.10-x86_64-linux-gnu/libpython3.10.a
target/lib/libpython3.10.a

(3) enable shared with static

git clean -fdx
./configure --with-static-python --enable-shared --prefix $PWD/target CFLAGS="-O0"
make && make install

Output:

$ ls -1 target/lib/python3.10/config-3.10-x86_64-linux-gnu/
config.c
config.c.in
install-sh
libpython3.10.a
Makefile
makesetup
__pycache__/
python-config.py
python.o
Setup
Setup.local

$ find target/ -name "libpython*"
target/lib/python3.10/config-3.10-x86_64-linux-gnu/libpython3.10.a
target/lib/libpython3.10.so.1.0
target/lib/libpython3.10.so
target/lib/libpython3.so

(4) enable shared without static

git clean -fdx
./configure --without-static-python --enable-shared --prefix $PWD/target CFLAGS="-O0"
make && make install

Output:

$ ls -1 target/lib/python3.10/config-3.10-x86_64-linux-gnu/
config.c
config.c.in
install-sh
Makefile
makesetup
__pycache__/
python-config.py
Setup
Setup.local

$ find target/ -name "libpython*"
target/lib/libpython3.10.so.1.0
target/lib/libpython3.10.so
target/lib/libpython3.so

=> target/lib/libpython3.10.a is not installed, as expected.

vstinner · 2021-02-02T23:47:52Z

(1) disable shared without static
target/lib/libpython3.10.a is still installed. Should it be not installed?

Update: I fixed the Makefile to not install libpython3.10.a in this case.

vstinner · 2021-02-03T00:30:14Z

Tests / Windows (x86) (pull_request) Failing after 18m — Windows (x86)

This failure is unrelated to my PR: "1 test failed: test_asyncio". (The Windows build is not affected by my PR.)

lazka · 2021-02-03T11:40:45Z

Misc/NEWS.d/next/Build/2021-02-02-16-26-44.bpo-43103.VWeyP_.rst

+By default, Python no longer builds ``libpythonMAJOR.MINOR.a`` static library
+and no longer installs it and ``python.o``. These files are only useful to
+build a static Python executable which is an uncommon use case. Add configure
+``--without-static-python`` option to build and install these files.


Should this read --with-static-python ?

Oh, the NEWS entry was outdated. I wrote the PR and the NEWS entry. Is it better now?

if static install is enabled by default now, then yes. thanks!

edit: I was slightly confused by the default change.. ignore my other comments.

vstinner · 2021-02-03T15:59:22Z

cc @pablogsal @hroncok

pablogsal · 2021-02-03T17:12:51Z

Doc/whatsnew/3.10.rst

@@ -797,6 +797,12 @@ Build Changes

  (Contributed by Victor Stinner in :issue:`42856`.)

+* Add configure ``--without-static-libpython`` option to not build
+  ``libpythonMAJOR.MINOR.a`` static library and not install ``python.o`` object


Suggested change

``libpythonMAJOR.MINOR.a`` static library and not install ``python.o`` object

``libpythonMAJOR.MINOR.a`` static library and not install the ``python.o`` object

Doc/whatsnew/3.10.rst

Misc/NEWS.d/next/Build/2021-02-02-16-26-44.bpo-43103.VWeyP_.rst

Add a new configure --without-static-libpython option to not build the libpythonMAJOR.MINOR.a static library and not install the python.o object file. Fix smelly.py and stable_abi.py tools when libpython3.10.a is missing.

vstinner · 2021-02-16T17:17:05Z

@pablogsal: Thanks, I took all your remarks in account. Would you mind to review the updated PR?

pablogsal · 2021-02-16T17:19:27Z

@pablogsal: Thanks, I took all your remarks in account. Would you mind to review the updated PR?

Yup, will re-review today

bedevere-bot · 2021-02-16T20:16:36Z

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit e1268bd 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

vstinner · 2021-02-16T20:54:02Z

buildbot/AMD64 FreeBSD Shared PR: Failure

This issue is known and unrelated to my PR: https://bugs.python.org/issue43233

vstinner · 2021-02-17T10:14:29Z

buildbot/AMD64 Windows10 PR: Failure

Oh, this failure is really strange.

 argv: ['Tools\\buildbot\\test.bat', '-p', 'x64', '-j4', '--junit-xml', 'test-results.xml', '--timeout', '900']
(...)
Windows fatal exception: stack overflow

Current thread 0x00002474 (most recent call first):
  File "D:\buildarea\pull_request.bolen-windows10\build\lib\_pyio.py", line 1619 in __repr__
  File "D:\buildarea\pull_request.bolen-windows10\build\lib\_pyio.py", line 1627 in __repr__
  File "D:\buildarea\pull_request.bolen-windows10\build\lib\_pyio.py", line 1627 in __repr__
(...)
0:03:22 load avg: 15.71 [ 59/426/3] test_exceptions crashed (Exit code 3221225725) -- running: test_zipfile (1 min 4 sec), test_tokenize (3 min 15 sec)
Windows fatal exception: stack overflow

Current thread 0x000023f4 (most recent call first):
  File "D:\buildarea\pull_request.bolen-windows10\build\lib\test\test_exceptions.py", line 585 in f
  File "D:\buildarea\pull_request.bolen-windows10\build\lib\test\test_exceptions.py", line 585 in f
  File "D:\buildarea\pull_request.bolen-windows10\build\lib\test\test_exceptions.py", line 585 in f
(...)
0:04:27 load avg: 14.79 [114/426/6] test_runpy crashed (Exit code 3221225725) -- running: test_socket (31.4 sec), test_asyncio (45.8 sec)
Windows fatal exception: stack overflow

Current thread 0x00002704 (most recent call first):
  File "<frozen zipimport>", line 343 in _get_module_path
  File "<frozen zipimport>", line 725 in _get_module_code
  File "<frozen zipimport>", line 223 in get_filename
(...)
17 tests failed:
    test_exceptions test_fileio test_io test_isinstance test_json
    test_lib2to3 test_logging test_pickle test_pickletools
    test_plistlib test_richcmp test_runpy test_sys test_threading
    test_traceback test_typing test_xml_etree
(...)

buildbot/aarch64 RHEL7 PR: Failure

The classic test_asyncio bug:

1 test altered the execution environment:
    test_asyncio

Add a new configure --without-static-libpython option to not build the libpythonMAJOR.MINOR.a static library and not install the python.o object file. Fix smelly.py and stable_abi.py tools when libpython3.10.a is missing.

edmorley · 2022-04-07T16:41:53Z

Hi! I don't suppose this would be eligible for a backport to Python 3.9? If so, would someone mind adding the needs backport to 3.9 label mentioned here?

vstinner · 2022-04-07T21:33:05Z

This is a new feature. We don't backport new features to stable branches sadly. https://devguide.python.org/#status-of-python-branches

edmorley · 2022-04-07T21:35:29Z

Ah ok thank you. Since this was build related, I thought there might be some wiggle room.

the-knights-who-say-ni added the CLA signed label Feb 2, 2021

bedevere-bot added the awaiting core review label Feb 2, 2021

lazka reviewed Feb 3, 2021

View reviewed changes

vstinner changed the title ~~bpo-43103: Don't install libpythonX.Y.a by default~~ bpo-43103: Add configure --without-static-libpython Feb 3, 2021

pablogsal reviewed Feb 3, 2021

View reviewed changes

Doc/whatsnew/3.10.rst Outdated Show resolved Hide resolved

pablogsal reviewed Feb 3, 2021

View reviewed changes

Misc/NEWS.d/next/Build/2021-02-02-16-26-44.bpo-43103.VWeyP_.rst Outdated Show resolved Hide resolved

bpo-43103: Add configure --without-static-libpython

e1268bd

Add a new configure --without-static-libpython option to not build the libpythonMAJOR.MINOR.a static library and not install the python.o object file. Fix smelly.py and stable_abi.py tools when libpython3.10.a is missing.

pablogsal approved these changes Feb 16, 2021

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Feb 16, 2021

pablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Feb 16, 2021

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Feb 16, 2021

vstinner merged commit 801bb0b into python:master Feb 17, 2021

bedevere-bot removed the awaiting merge label Feb 17, 2021

vstinner deleted the static_opt branch February 17, 2021 10:14

edmorley mentioned this pull request May 3, 2022

Binary build script: Use shared builds + LTO for Python 3.10 heroku/heroku-buildpack-python#1320

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-43103: Add configure --without-static-libpython #24418

bpo-43103: Add configure --without-static-libpython #24418

vstinner commented Feb 2, 2021 •

edited

Loading

vstinner commented Feb 2, 2021

vstinner commented Feb 2, 2021 •

edited

Loading

vstinner commented Feb 3, 2021

lazka Feb 3, 2021 •

edited

Loading

vstinner Feb 3, 2021

lazka Feb 3, 2021 •

edited

Loading

vstinner commented Feb 3, 2021

pablogsal Feb 3, 2021

vstinner commented Feb 16, 2021

pablogsal commented Feb 16, 2021

bedevere-bot commented Feb 16, 2021

vstinner commented Feb 16, 2021

vstinner commented Feb 17, 2021

edmorley commented Apr 7, 2022

vstinner commented Apr 7, 2022

edmorley commented Apr 7, 2022

	``libpythonMAJOR.MINOR.a`` static library and not install ``python.o`` object
	``libpythonMAJOR.MINOR.a`` static library and not install the ``python.o`` object

bpo-43103: Add configure --without-static-libpython #24418

bpo-43103: Add configure --without-static-libpython #24418

Conversation

vstinner commented Feb 2, 2021 • edited Loading

vstinner commented Feb 2, 2021

(1) disable shared without static

(2) disable shared with static

(3) enable shared with static

(4) enable shared without static

vstinner commented Feb 2, 2021 • edited Loading

vstinner commented Feb 3, 2021

lazka Feb 3, 2021 • edited Loading

Choose a reason for hiding this comment

vstinner Feb 3, 2021

Choose a reason for hiding this comment

lazka Feb 3, 2021 • edited Loading

Choose a reason for hiding this comment

vstinner commented Feb 3, 2021

pablogsal Feb 3, 2021

Choose a reason for hiding this comment

vstinner commented Feb 16, 2021

pablogsal commented Feb 16, 2021

bedevere-bot commented Feb 16, 2021

vstinner commented Feb 16, 2021

vstinner commented Feb 17, 2021

edmorley commented Apr 7, 2022

vstinner commented Apr 7, 2022

edmorley commented Apr 7, 2022

vstinner commented Feb 2, 2021 •

edited

Loading

vstinner commented Feb 2, 2021 •

edited

Loading

lazka Feb 3, 2021 •

edited

Loading

lazka Feb 3, 2021 •

edited

Loading