Move main functions to a separate Programs directory

[ncoghlan]

BPO	18093
Nosy	@warsaw, @ncoghlan, @vstinner, @ezio-melotti, @ericsnowcurrently, @zware, @sdrees
Files	move_programs.git.diff: hg export --git test_capi.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-09-25.09:49:27.612>
created_at = <Date 2013-05-29.11:51:30.150>
labels = ['interpreter-core', 'type-feature', 'build']
title = 'Move main functions to a separate Programs directory'
updated_at = <Date 2014-09-25.09:49:27.611>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2014-09-25.09:49:27.611>
actor = 'ncoghlan'
assignee = 'none'
closed = True
closed_date = <Date 2014-09-25.09:49:27.612>
closer = 'ncoghlan'
components = ['Build', 'Interpreter Core']
creation = <Date 2013-05-29.11:51:30.150>
creator = 'ncoghlan'
dependencies = []
files = ['30409', '36720']
hgrepos = []
issue_num = 18093
keywords = ['patch']
message_count = 14.0
messages = ['190292', '190408', '192267', '192268', '223936', '223937', '223939', '223941', '223953', '223989', '227513', '227514', '227518', '227519']
nosy_count = 10.0
nosy_names = ['barry', 'ncoghlan', 'vstinner', 'ezio.melotti', 'Arfrever', 'tshepang', 'python-dev', 'eric.snow', 'zach.ware', 'dilettant']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue18093'
versions = ['Python 3.4']

[ncoghlan]

This patch moves the C level main functions for python, _testembed and _freeze_importlib to a new Programs directory.

I added README files with my current understanding of the expected contents of the Python, Objects, Modules and Programs directories and also attempted to update the Windows builds (which don't appear to include _testembed at all).

[zware]

I can confirm that the patch doesn't break building on Windows.

Would it make any sense to move Windows-specific sources for things like kill_python.exe (PCbuild/kill_python.c), make_buildinfo.exe, make_versioninfo.exe, py.exe (PC/launcher.c) into Programs? Or better to keep them in PC or PCbuild (at least for now, until after this patch is approved)?

[ericsnowcurrently]

Looks good to me. What else needs to be done in this area that will help with work on PEP-432?

[ericsnowcurrently]

What about Modules/main.c?

[python-dev]

New changeset 88a532a31eb3 by Nick Coghlan in branch 'default':
Issue bpo-18093: Factor out the programs that embed the runtime
http://hg.python.org/cpython/rev/88a532a31eb3

[python-dev]

New changeset 7640af73c19d by Nick Coghlan in branch 'default':
Add missing NEWS entry for issue bpo-18093
http://hg.python.org/cpython/rev/7640af73c19d

[ncoghlan]

Prompted to finally get back to this by the python-dev discussion about PEP-432 (as separating these out helps keep the moving parts clear when working on the startup sequence)

In response to Eric's question, Modules/main.c is part of the CPython runtime, rather than a C application that embeds the runtime. The "main" in the name refers to the exported Py_Main, rather than a C main function.

The *actual* C level main function now lives in Programs/python.c (previously Modules/python.c) and calls Py_Main after doing a bit of fiddling about with the command line arguments and the floating point settings.

[ncoghlan]

Zach, regarding the Windows executables - up to you if you want to open an issue to move them. These ones I particularly wanted to move because I found having them in Modules to be genuinely confusing when working on the startup code.

[python-dev]

New changeset 766570a5d607 by Zachary Ware in branch 'default':
Issue bpo-18093: Give the Windows build _testembed.c's new location.
http://hg.python.org/cpython/rev/766570a5d607

[python-dev]

New changeset 413017768dde by Ned Deily in branch 'default':
Issue bpo-18093: Create Programs directory in build location for
http://hg.python.org/cpython/rev/413017768dde

[vstinner]

_testembed was moved from Modules to Programs but test_capi was not updated: it still looks for test_embed in Modules.

$ ./configure --with-pydebug
$ make
$ ./python -m test -v test_capi
...
test_forced_io_encoding (test.test_capi.EmbeddingTests) ... skipped "'/home/haypo/prog/python/default/Modules/_testembed' doesn't exist"
test_subinterps (test.test_capi.EmbeddingTests) ... skipped "'/home/haypo/prog/python/default/Modules/_testembed' doesn't exist"
...

[vstinner]

test_capi.patch fixes test_capi to be able to run _testembed in test_capi.

[python-dev]

New changeset c87e00a6258d by Nick Coghlan in branch 'default':
Issue bpo-18093: fix test_capi test skip due to _testembed move
https://hg.python.org/cpython/rev/c87e00a6258d

[ncoghlan]

Thanks for spotting that Victor - should be fixed by that last commit with your change.