Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Race condition during installation in parallel builds #104692

Closed
jefferyto opened this issue May 20, 2023 · 1 comment
Closed

Race condition during installation in parallel builds #104692

jefferyto opened this issue May 20, 2023 · 1 comment
Labels
build The build process and cross-build type-bug An unexpected behavior, bug, or error

Comments

@jefferyto
Copy link
Contributor

jefferyto commented May 20, 2023 •
edited by bedevere-bot

Bug report

When running make -j install, it is possible for the python3 symlink to be installed before all standard library modules are installed. If an external program tries to run Python at this moment it can lead to an execution error.

We are getting sporadic build errors at OpenWrt (openwrt/packages#19241) and believe this to be the cause, that during the build for "host" Python (Python that runs on the buildbot), another package being built tries to run a script using this Python because the python3 symlink is in place.

Your environment

  • CPython versions tested on: 3.10
  • Operating system and architecture: Linux, not sure exactly which distribution

Linked PRs
@jefferyto jefferyto added the type-bug An unexpected behavior, bug, or error label May 20, 2023
@erlend-aasland erlend-aasland added the build The build process and cross-build label May 20, 2023
jefferyto added a commit to jefferyto/cpython that referenced this issue May 20, 2023
@jefferyto
pythongh-104692: Include commoninstall as a prerequisite for bininstall
3a09491 
This ensures that commoninstall is completed before bininstall is
started when parallel builds are used (make -j install), and so the
python3 symlink is only installed after all standard library modules are
installed.
@bedevere-bot bedevere-bot mentioned this issue May 20, 2023
Merged
jefferyto added a commit to jefferyto/cpython that referenced this issue May 20, 2023
@jefferyto
pythongh-104692: Include commoninstall as a prerequisite for bininstall
8859645 
This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.
jefferyto added a commit to jefferyto/cpython that referenced this issue May 20, 2023
@jefferyto
pythongh-104692: Include commoninstall as a prerequisite for bininstall
869cd18 
This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.
ambv pushed a commit that referenced this issue Jun 7, 2023
@jefferyto @zware
gh-104692: Include commoninstall as a prerequisite for bininstall (#1…
990cb36 
…04693)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
jefferyto added a commit to jefferyto/cpython that referenced this issue Jun 7, 2023
@jefferyto
[3.12] pythongh-104692: Include commoninstall as a prerequisite for b…
c4c7ffe 
…ininstall (pythonGH-104693)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
(cherry picked from commit 990cb36)

Co-authored-by: Jeffery To <jeffery.to@gmail.com>
This was referenced Jun 7, 2023
Merged
Merged
jefferyto added a commit to jefferyto/cpython that referenced this issue Jun 7, 2023
@jefferyto
[3.11] pythongh-104692: Include commoninstall as a prerequisite for b…
94d5d97 
…ininstall (pythonGH-104693)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

Co-authored-by: Zachary Ware <zachary.ware@gmail.com>.
(cherry picked from commit 990cb36)

Co-authored-by: Jeffery To <jeffery.to@gmail.com>
ambv pushed a commit that referenced this issue Jul 5, 2023
@jefferyto
[3.12] gh-104692: Include commoninstall as a prerequisite for bininst…
2edec6a 
…all (GH-104693) (#105428)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

(cherry picked from commit 990cb36)
ambv pushed a commit that referenced this issue Jul 5, 2023
@jefferyto @erlend-aasland
[3.11] gh-104692: Include commoninstall as a prerequisite for bininst…
6c6a2a9 
…all (GH-104693) (#105429)

This ensures that `commoninstall` is completed before `bininstall` is
started when parallel builds are used (`make -j install`), and so the
`python3` symlink is only installed after all standard library modules
are installed.

.
(cherry picked from commit 990cb36)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
@jefferyto
Copy link
Contributor Author

Fixes have been merged, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build The build process and cross-build type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jefferyto @erlend-aasland