bpo-38662: Invoke pip via runpy, in ensurepip #18901

hroncok · 2020-03-10T14:34:04Z

This way, any changes to the internal pip API won't mean ensurepip needs to be
changed as well. Also, distributors can update their pip wheels independent on
CPython release schedule.

Co-Authored-By: @pradyunsg

Closes #17029

https://bugs.python.org/issue38662

This way, any changes to the internal pip API won't mean ensurepip needs to be changed as well. Also, distributors can update their pip wheels independent on CPython release schedule. Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com> Co-Authored-By: Miro Hrončok <miro@hroncok.cz>

Lib/ensurepip/__init__.py

Co-Authored-By: Victor Stinner <vstinner@python.org>

Lib/ensurepip/__init__.py

pradyunsg

Looks good to go, to me!

Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com>

vstinner · 2020-03-10T21:17:47Z

Thanks @hroncok and @pradyunsg.

While it's not a major issue in CPython, I can say that being tightly coupled to pip internals is a major in Fedora which updates pip frequently and tries to support multiple pip versions.

miss-islington · 2020-03-10T21:18:07Z

Thanks @hroncok for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

miss-islington · 2020-03-10T21:18:08Z

Thanks @hroncok for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

The ensurepip module now invokes pip via the runpy module. Hence it is no longer tightly coupled with the internal API of the bundled pip version, allowing easier updates to a newer pip version both internally and for distributors. This way, any changes to the internal pip API won't mean ensurepip needs to be changed as well. Also, distributors can update their pip wheels independent on CPython release schedule. Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com> Co-Authored-By: Miro Hrončok <miro@hroncok.cz> (cherry picked from commit 88f82b2) Co-authored-by: Miro Hrončok <miro@hroncok.cz>

bedevere-bot · 2020-03-10T21:18:23Z

GH-18906 is a backport of this pull request to the 3.8 branch.

bedevere-bot · 2020-03-10T21:18:31Z

GH-18907 is a backport of this pull request to the 3.7 branch.

The ensurepip module now invokes pip via the runpy module. Hence it is no longer tightly coupled with the internal API of the bundled pip version, allowing easier updates to a newer pip version both internally and for distributors. This way, any changes to the internal pip API won't mean ensurepip needs to be changed as well. Also, distributors can update their pip wheels independent on CPython release schedule. Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com> Co-Authored-By: Miro Hrončok <miro@hroncok.cz> (cherry picked from commit 88f82b2) Co-authored-by: Miro Hrončok <miro@hroncok.cz>

hroncok · 2020-03-10T21:23:39Z

I wasn't even hoping for a backport. Thanks!

The ensurepip module now invokes pip via the runpy module. Hence it is no longer tightly coupled with the internal API of the bundled pip version, allowing easier updates to a newer pip version both internally and for distributors. This way, any changes to the internal pip API won't mean ensurepip needs to be changed as well. Also, distributors can update their pip wheels independent on CPython release schedule. Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com> Co-Authored-By: Miro Hrončok <miro@hroncok.cz> (cherry picked from commit 88f82b2) Co-authored-by: Miro Hrončok <miro@hroncok.cz>

vstinner · 2020-03-12T14:24:33Z

@petdance: Do you have an idea why your CPython fork creates almost 20 new " petdance added a commit to petdance/cpython that referenced this pull request" comments???

I see that https://github.com/petdance/cpython/commit/c861affb02eb7a4a90d7c34f86c743545551d8a4 commit starts with " bpo-38662: ensurepip invokes pip via runpy (GH-18901) ". What is this "(GH-18901)" syntax? do you rewrite the Python history?

petdance · 2020-03-12T14:59:02Z

I'm sorry, I don't know what's happening.

My normal process is to pull from upstream master daily, and then rebase my working branches against it my new master. Then I push the branches back. I'm not changing any history. I don't know what the "GH-18901" syntax is.

Clearly c861affb02eb7a4a90d7c34f86c743545551d8a4 somehow got created on my calloc branch but I don't know how/why. I'll look into it tonight.

vstinner · 2020-03-12T16:27:35Z

In the master branch, the "bpo-38662: ensurepip invokes pip via runpy (GH-18901)" has the sha1: 88f82b2. In your repository, it has the sha1: c861affb02eb7a4a90d7c34f86c743545551d8a4 and the title "bpo-38662: ensurepip invokes pip via runpy (GH-18901)". Something replaced "GH-18901" with "GH-18901".

"GH-18901" syntax is used in Python to mean "identifier 18901 in the GitHub namespace", as bpo-123 is "123 in the bugs.python.org namespace".

My normal process is to pull from upstream master daily, and then rebase my working branches against it my new master.

How do you pull commits from GitHub? Which tools/commands are you using?

petdance · 2020-03-12T16:33:31Z

git co master
git pull upstream master
git push
git co dev
git rebase master
git push

That's it.

Maybe I screwed up a commit on my one calloc branch. Or has this happened elsewhere that can give me a clue?

vstinner · 2020-03-12T16:41:57Z

Maybe I screwed up a commit on my one calloc branch. Or has this happened elsewhere that can give me a clue?

Try maybe to clone again a fresh copy of the cpython Git repository and restart from that point.

petdance · 2020-03-13T00:42:39Z

I deleted my calloc branch and recreated it fresh, and pushed it back to github. And then I git gcd my local repo. git branch --contains no longer finds the commit, but it's still on the Github side. Please let me know if this happens again. Sorry about the mess.

vstinner · 2020-03-13T07:43:12Z

You added two more commmits to this PR :-)

petdance added a commit to petdance/cpython that referenced this pull request 4 hours ago

and

petdance added a commit to petdance/cpython that referenced this pull request 4 hours ago

"I deleted my calloc branch" Aha. I added PyMem_Calloc(). I recall that I ran some benchmarks. Tell me if you want me to have a look at your PR.

petdance · 2020-03-13T14:25:59Z

The two weird commits have nothing to do with the calloc branch, though. :-(

Is there a bpo ticket you'd like me to add a calloc PR to? It's just 16 places where I changed malloc+memset to calloc.

vstinner · 2020-03-13T16:19:18Z

Is there a bpo ticket you'd like me to add a calloc PR to? It's just 16 places where I changed malloc+memset to calloc.

If you want to make changes related to calloc, please open a new issue. I'm not aware of an open issue related to calloc.

hroncok · 2020-03-13T16:24:02Z

JFYI I've unsubscribed. If there is anything related to this PR itself that needs my attention, please do mention me.

the-knights-who-say-ni added the CLA signed label Mar 10, 2020

bedevere-bot added the awaiting review label Mar 10, 2020

hroncok mentioned this pull request Mar 10, 2020

bpo-38662: Invoke pip via runpy, in ensurepip #17029

Closed

hroncok force-pushed the use-runpy-in-ensurepip branch from 8b5b564 to f8008ba Compare March 10, 2020 14:36

vstinner reviewed Mar 10, 2020

View reviewed changes

Lib/ensurepip/__init__.py Outdated Show resolved Hide resolved

Lib/ensurepip/__init__.py Show resolved Hide resolved

Lib/ensurepip/__init__.py Outdated Show resolved Hide resolved

Apply suggestions from code review

c4b431a

Co-Authored-By: Victor Stinner <vstinner@python.org>

hroncok commented Mar 10, 2020

View reviewed changes

Lib/ensurepip/__init__.py Outdated Show resolved Hide resolved

Typo

1109d5b

pradyunsg reviewed Mar 10, 2020

View reviewed changes

Lib/ensurepip/__init__.py Outdated Show resolved Hide resolved

pradyunsg approved these changes Mar 10, 2020

View reviewed changes

bedevere-bot added awaiting core review and removed awaiting review labels Mar 10, 2020

Translate the exception message to English

1db1b1d

Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com>

vstinner merged commit 88f82b2 into python:master Mar 10, 2020

bedevere-bot removed the awaiting core review label Mar 10, 2020

vstinner added needs backport to 3.7 labels Mar 10, 2020

bedevere-bot removed the needs backport to 3.8 label Mar 10, 2020

bedevere-bot removed the needs backport to 3.7 label Mar 10, 2020

hroncok deleted the use-runpy-in-ensurepip branch March 10, 2020 21:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-38662: Invoke pip via runpy, in ensurepip #18901

bpo-38662: Invoke pip via runpy, in ensurepip #18901

hroncok commented Mar 10, 2020 •

edited by bedevere-bot

Loading

pradyunsg left a comment

vstinner commented Mar 10, 2020

miss-islington commented Mar 10, 2020

miss-islington commented Mar 10, 2020

bedevere-bot commented Mar 10, 2020

bedevere-bot commented Mar 10, 2020

hroncok commented Mar 10, 2020

vstinner commented Mar 12, 2020 •

edited by bedevere-bot

Loading

petdance commented Mar 12, 2020

vstinner commented Mar 12, 2020 •

edited by bedevere-bot

Loading

petdance commented Mar 12, 2020 •

edited

Loading

vstinner commented Mar 12, 2020

petdance commented Mar 13, 2020 •

edited

Loading

vstinner commented Mar 13, 2020

petdance commented Mar 13, 2020

vstinner commented Mar 13, 2020

hroncok commented Mar 13, 2020

bpo-38662: Invoke pip via runpy, in ensurepip #18901

bpo-38662: Invoke pip via runpy, in ensurepip #18901

Conversation

hroncok commented Mar 10, 2020 • edited by bedevere-bot Loading

pradyunsg left a comment

Choose a reason for hiding this comment

vstinner commented Mar 10, 2020

miss-islington commented Mar 10, 2020

miss-islington commented Mar 10, 2020

bedevere-bot commented Mar 10, 2020

bedevere-bot commented Mar 10, 2020

hroncok commented Mar 10, 2020

vstinner commented Mar 12, 2020 • edited by bedevere-bot Loading

petdance commented Mar 12, 2020

vstinner commented Mar 12, 2020 • edited by bedevere-bot Loading

petdance commented Mar 12, 2020 • edited Loading

vstinner commented Mar 12, 2020

petdance commented Mar 13, 2020 • edited Loading

vstinner commented Mar 13, 2020

petdance commented Mar 13, 2020

vstinner commented Mar 13, 2020

hroncok commented Mar 13, 2020

hroncok commented Mar 10, 2020 •

edited by bedevere-bot

Loading

vstinner commented Mar 12, 2020 •

edited by bedevere-bot

Loading

vstinner commented Mar 12, 2020 •

edited by bedevere-bot

Loading

petdance commented Mar 12, 2020 •

edited

Loading

petdance commented Mar 13, 2020 •

edited

Loading