Add pip inspect command #11245

sbidoul · 2022-07-10T17:42:47Z

This follows from the discussions in #11223 and #11097. Inspired from pip list and the installation report format.

This is fairly straightforward (although the underlying logic in pip may not be as trivial as it looks at first glance).

Fully open to bike shedding and everything of course.

@pypa/pip-committers if you agree (or don't disagree too much :) I can document it in the same style as #10771, and this can be done fairly quickly.

closes #5261
closes #7967
closes #8008
closes #11223

Example:

{
  "version": "0",
  "pip_version": "22.2.dev0",
  "installed": [
    {
      "metadata": {
        "metadata_version": "2.1",
        "name": "pyparsing",
        "version": "3.0.9",
        "summary": "pyparsing module - Classes and methods to define and execute parsing grammars",
        "description_content_type": "text/x-rst",
        "author_email": "Paul McGuire <ptmcg.gm+pyparsing@gmail.com>",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Intended Audience :: Developers",
          "Intended Audience :: Information Technology",
          "License :: OSI Approved :: MIT License",
          "Operating System :: OS Independent",
          "Programming Language :: Python",
          "Programming Language :: Python :: 3",
          "Programming Language :: Python :: 3.6",
          "Programming Language :: Python :: 3.7",
          "Programming Language :: Python :: 3.8",
          "Programming Language :: Python :: 3.9",
          "Programming Language :: Python :: 3.10",
          "Programming Language :: Python :: 3 :: Only",
          "Programming Language :: Python :: Implementation :: CPython",
          "Programming Language :: Python :: Implementation :: PyPy",
          "Typing :: Typed"
        ],
        "requires_dist": [
          "railroad-diagrams ; extra == \"diagrams\"",
          "jinja2 ; extra == \"diagrams\""
        ],
        "requires_python": ">=3.6.8",
        "project_url": [
          "Homepage, https://github.com/pyparsing/pyparsing/"
        ],
        "provides_extra": [
          "diagrams"
        ],
        "description": "..."
      },
      "metadata_location": "/home/me/.virtualenvs/demoenv/lib/python3.8/site-packages/pyparsing-3.0.9.dist-info",
      "installer": "pip",
      "requested": false
    },
    {
      "metadata": {
        "metadata_version": "2.1",
        "name": "packaging",
        "version": "21.3",
        "platform": [
          "UNKNOWN"
        ],
        "summary": "Core utilities for Python packages",
        "description_content_type": "text/x-rst",
        "home_page": "https://github.com/pypa/packaging",
        "author": "Donald Stufft and individual contributors",
        "author_email": "donald@stufft.io",
        "license": "BSD-2-Clause or Apache-2.0",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: Apache Software License",
          "License :: OSI Approved :: BSD License",
          "Programming Language :: Python",
          "Programming Language :: Python :: 3",
          "Programming Language :: Python :: 3 :: Only",
          "Programming Language :: Python :: 3.6",
          "Programming Language :: Python :: 3.7",
          "Programming Language :: Python :: 3.8",
          "Programming Language :: Python :: 3.9",
          "Programming Language :: Python :: 3.10",
          "Programming Language :: Python :: Implementation :: CPython",
          "Programming Language :: Python :: Implementation :: PyPy"
        ],
        "requires_dist": [
          "pyparsing (!=3.0.5,>=2.0.2)"
        ],
        "requires_python": ">=3.6",
        "description": "..."
      },
      "metadata_location": "/home/me/.virtualenvs/demoenv/lib/python3.8/site-packages/packaging-21.3.dist-info",
      "installer": "pip",
      "requested": true
    },
    {
      "metadata": {
        "metadata_version": "2.1",
        "name": "pip",
        "version": "22.2.dev0",
        "summary": "The PyPA recommended tool for installing Python packages.",
        "home_page": "https://pip.pypa.io/",
        "author": "The pip developers",
        "author_email": "distutils-sig@python.org",
        "license": "MIT",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: MIT License",
          "Topic :: Software Development :: Build Tools",
          "Programming Language :: Python",
          "Programming Language :: Python :: 3",
          "Programming Language :: Python :: 3 :: Only",
          "Programming Language :: Python :: 3.7",
          "Programming Language :: Python :: 3.8",
          "Programming Language :: Python :: 3.9",
          "Programming Language :: Python :: 3.10",
          "Programming Language :: Python :: Implementation :: CPython",
          "Programming Language :: Python :: Implementation :: PyPy"
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Documentation, https://pip.pypa.io",
          "Source, https://github.com/pypa/pip",
          "Changelog, https://pip.pypa.io/en/stable/news/"
        ],
        "description": "..."
      },
      "metadata_location": "/home/me/pip/src/pip.egg-info",
      "direct_url": {
        "url": "file:///home/me/pip/src",
        "dir_info": {
          "editable": true
        }
      }
    }
  ],
  "environment": {
    "implementation_name": "cpython",
    "implementation_version": "3.8.10",
    "os_name": "posix",
    "platform_machine": "x86_64",
    "platform_release": "5.13-generic",
    "platform_system": "Linux",
    "platform_version": "...",
    "python_full_version": "3.8.10",
    "platform_python_implementation": "CPython",
    "python_version": "3.8",
    "sys_platform": "linux"
  }
}

pradyunsg · 2022-07-10T21:33:34Z

I’m on board, and pleasantly surprised at how quickly you turned this around! We’re having a very active weekend across PyPI and pip. :)

docs/html/reference/inspect-report.md

src/pip/_internal/commands/inspect.py

sbidoul · 2022-07-12T17:35:30Z

I marked this issue as closing the pip show --json issue and siblings. It is not exactly the same but it is easy to use pip inspect and filter out a single distribution out of its result.

@pradyunsg I found where you mentioned pip inspect: #8077 (comment). This pip inspect is not quite what you had in mind back then, I guess.

But then my stance wrt feature request such as include pip-deptree in pip, would be to say we don't want these in pip itself, but that, thanks to pip inspect (this PR) + packaging, it should be quite easy to update such a tool so that

it does not need to be installed in the target environment (i.e. it can be installed with pipx instead)
it does not need to access pip internals (as it can call pip inspect to get what is installed and then process it with the packaging library).

Any opinion about that, @pypa/pip-committers ?

pradyunsg · 2022-07-13T08:24:33Z

I think the whole idea there, is that you shouldn't need to install anything extra, anywhere, to get dependency information in that format.

I think it's a reasonable enhancement to make and being able to accommodate for such needs, around inspecting the installation database and viewing it in different ways, was partly the motivation in suggesting that this have subcommands.

I'm almost certainly in the camp of "let's not back ourselves into a corner with the design choices we make today", but I empathise that it often comes with additional complexity in the initial implementation to design for the extensibility involved.

pradyunsg · 2022-07-13T08:35:22Z

But, anyway, I'm not married to that idea and making this information available in a manner that other tooling can build off of pip is a good design approach that I'd say we can do more of.

sbidoul · 2022-07-13T19:50:47Z

I added a test and documentation. This should be ready.

Tentatively adding to 22.2 as it would make sense to announce this together with the installation report, since they are complementary.

uranusjr · 2022-07-15T08:26:13Z

docs/html/reference/inspect-report.md

+
+- `version`: the string `0`, denoting that the inspect command is an experimental
+  feature. This value will change to `1`, when the feature is deemed stable after
+  gathering user feedback (likely in pip 22.3 or 23.0). Backward incompatible changes


I wonder if we should commit to this… Versions numbers recorded in documentation likely this generally get stale without being updated.

I think its good to give a sense of when we intend the feature to become stable, and I'm motivated to follow this up in the coming month so it's unlikely that I'll forget. So I'd be in favor of keeping it.

And, we're using "likely" to give us some leeway, in case we do miss this deadline in some way. :)

src/pip/_internal/metadata/base.py

src/pip/_internal/commands/inspect.py

docs/html/cli/pip_inspect.rst

Bumps [pip](https://github.com/pypa/pip) from 22.1.2 to 22.2. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p> <blockquote> <h1>22.2 (2022-07-21)</h1> <h2>Deprecations and Removals</h2> <ul> <li>Remove the <code>html5lib</code> deprecated feature flag. (<code>[#10825](pypa/pip#10825) <https://github.com/pypa/pip/issues/10825></code>_)</li> <li>Remove <code>--use-deprecated=backtrack-on-build-failures</code>. (<code>[#11241](pypa/pip#11241) <https://github.com/pypa/pip/issues/11241></code>_)</li> </ul> <h2>Features</h2> <ul> <li> <p>Add support to use <code>truststore <https://pypi.org/project/truststore/></code>_ as an alternative SSL certificate verification backend. The backend can be enabled on Python 3.10 and later by installing <code>truststore</code> into the environment, and adding the <code>--use-feature=truststore</code> flag to various pip commands.</p> <p><code>truststore</code> differs from the current default verification backend (provided by <code>certifi</code>) in it uses the operating system’s trust store, which can be better controlled and augmented to better support non-standard certificates. Depending on feedback, pip may switch to this as the default certificate verification backend in the future. (<code>[#11082](pypa/pip#11082) <https://github.com/pypa/pip/issues/11082></code>_)</p> </li> <li> <p>Add <code>--dry-run</code> option to <code>pip install</code>, to let it print what it would install but not actually change anything in the target environment. (<code>[#11096](pypa/pip#11096) <https://github.com/pypa/pip/issues/11096></code>_)</p> </li> <li> <p>Record in wheel cache entries the URL of the original artifact that was downloaded to build the cached wheels. The record is named <code>origin.json</code> and uses the PEP 610 Direct URL format. (<code>[#11137](pypa/pip#11137) <https://github.com/pypa/pip/issues/11137></code>_)</p> </li> <li> <p>Support <code>PEP 691 <https://peps.python.org/pep-0691/></code><em>. (<code>[#11158](pypa/pip#11158) <https://github.com/pypa/pip/issues/11158></code></em>)</p> </li> <li> <p>pip's deprecation warnings now subclass the built-in <code>DeprecationWarning</code>, and can be suppressed by running the Python interpreter with <code>-W ignore::DeprecationWarning</code>. (<code>[#11225](pypa/pip#11225) <https://github.com/pypa/pip/issues/11225></code>_)</p> </li> <li> <p>Add <code>pip inspect</code> command to obtain the list of installed distributions and other information about the Python environment, in JSON format. (<code>[#11245](pypa/pip#11245) <https://github.com/pypa/pip/issues/11245></code>_)</p> </li> <li> <p>Significantly speed up isolated environment creation, by using the same sources for pip instead of creating a standalone installation for each environment. (<code>[#11257](pypa/pip#11257) <https://github.com/pypa/pip/issues/11257></code>_)</p> </li> <li> <p>Add an experimental <code>--report</code> option to the install command to generate a JSON report of what was installed. In combination with <code>--dry-run</code> and <code>--ignore-installed</code> it can be used to resolve the requirements. (<code>[#53](pypa/pip#53) <https://github.com/pypa/pip/issues/53></code>_)</p> </li> </ul> <h2>Bug Fixes</h2> <ul> <li>Fix <code>pip install --pre</code> for packages with pre-release build dependencies defined both in <code>pyproject.toml</code>'s <code>build-system.requires</code> and <code>setup.py</code>'s <code>setup_requires</code>. (<code>[#10222](pypa/pip#10222) <https://github.com/pypa/pip/issues/10222></code>_)</li> <li>When pip rewrites the shebang line in a script during wheel installation, update the hash and size in the corresponding <code>RECORD</code> file entry. (<code>[#10744](pypa/pip#10744) <https://github.com/pypa/pip/issues/10744></code>_)</li> <li>Do not consider a <code>.dist-info</code> directory found inside a wheel-like zip file as metadata for an installed distribution. A package in a wheel is (by</li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/pip/commit/8e7e76e60f4e115ea1201bee2f176377a718fce1"><code>8e7e76e</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/b6f6a94e36f10a4535ea5bbdc6b351f62003eede"><code>b6f6a94</code></a> Update AUTHORS.txt</li> <li><a href="https://github.com/pypa/pip/commit/790725aca3f60c745e33827a6079d9600da373d8"><code>790725a</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11274">#11274</a> from sbidoul/install-report-note-sbi</li> <li><a href="https://github.com/pypa/pip/commit/d4b9e187aa7cc5ab14b2339f6171f7f2ea6504e9"><code>d4b9e18</code></a> Add clarifications to the installation report documentation</li> <li><a href="https://github.com/pypa/pip/commit/b1a01ef762a78af1194958a1c874015eaf81fd04"><code>b1a01ef</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11265">#11265</a> from finnagin/main</li> <li><a href="https://github.com/pypa/pip/commit/48bcb0a4ccd30a9d00e58fe58827772e307a7e39"><code>48bcb0a</code></a> reformat to pass pre-commit check</li> <li><a href="https://github.com/pypa/pip/commit/a7c1fe3bff5655393018c53b448b669b3525515b"><code>a7c1fe3</code></a> Remove utc fixture from tests</li> <li><a href="https://github.com/pypa/pip/commit/0c574f72905185d62bcca741c813df9bae1d9282"><code>0c574f7</code></a> Remove time import</li> <li><a href="https://github.com/pypa/pip/commit/246fef19149eea893f1cf3efd53f9b17c94c952f"><code>246fef1</code></a> Remove utc fixture</li> <li><a href="https://github.com/pypa/pip/commit/c9cb7f4629bdd8c61b792feff6dacb1d2e848d57"><code>c9cb7f4</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/pip/issues/11270">#11270</a> from uranusjr/upgrade-pre-commit-hooks</li> <li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/22.1.2...22.2">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=22.1.2&new-version=22.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

sbidoul mentioned this pull request Jul 10, 2022

Add metadata field to pip list json output #11097

Closed

sbidoul force-pushed the pip-inspect-sbi branch 2 times, most recently from 8ba7e47 to 9bc4dca Compare July 10, 2022 17:48

sbidoul force-pushed the pip-inspect-sbi branch 2 times, most recently from 248ed1b to 28993f0 Compare July 11, 2022 20:43

pradyunsg reviewed Jul 11, 2022

View reviewed changes

docs/html/reference/inspect-report.md Outdated Show resolved Hide resolved

docs/html/reference/inspect-report.md Show resolved Hide resolved

src/pip/_internal/commands/inspect.py Outdated Show resolved Hide resolved

src/pip/_internal/commands/inspect.py Show resolved Hide resolved

sbidoul force-pushed the pip-inspect-sbi branch 4 times, most recently from 29e7a09 to 40f2a54 Compare July 12, 2022 17:06

sbidoul force-pushed the pip-inspect-sbi branch from 40f2a54 to 7b444b0 Compare July 13, 2022 08:31

sbidoul force-pushed the pip-inspect-sbi branch from 7b444b0 to b5aff24 Compare July 13, 2022 16:53

sbidoul marked this pull request as ready for review July 13, 2022 19:49

sbidoul added this to the 22.2 milestone Jul 13, 2022

sbidoul force-pushed the pip-inspect-sbi branch 2 times, most recently from 3b9abdb to 0ce7f39 Compare July 14, 2022 19:07

sbidoul requested a review from a team July 15, 2022 08:14

uranusjr reviewed Jul 15, 2022

View reviewed changes

src/pip/_internal/metadata/base.py Outdated Show resolved Hide resolved

uranusjr reviewed Jul 15, 2022

View reviewed changes

src/pip/_internal/commands/inspect.py Outdated Show resolved Hide resolved

sbidoul force-pushed the pip-inspect-sbi branch from 0ce7f39 to 0d377cf Compare July 15, 2022 08:40

sbidoul added the type: enhancement Improvements to functionality label Jul 15, 2022

github-actions bot added the needs rebase or merge PR has conflicts with current master label Jul 15, 2022

Add requested property to BaseDistribution

7f46f94

sbidoul force-pushed the pip-inspect-sbi branch from 0d377cf to 9ff66f6 Compare July 15, 2022 10:37

pypa-bot removed the needs rebase or merge PR has conflicts with current master label Jul 15, 2022

pradyunsg reviewed Jul 15, 2022

View reviewed changes

docs/html/cli/pip_inspect.rst Show resolved Hide resolved

sbidoul force-pushed the pip-inspect-sbi branch from 9ff66f6 to 730e597 Compare July 15, 2022 12:02

sbidoul added 3 commits July 15, 2022 14:08

Add pip inspect command

5851364

inspect: add experimental status warning

a7df613

inspect: add versionadded to docs

44d340a

sbidoul force-pushed the pip-inspect-sbi branch from 730e597 to 44d340a Compare July 15, 2022 12:08

sbidoul requested review from pradyunsg and uranusjr July 16, 2022 17:48

uranusjr approved these changes Jul 17, 2022

View reviewed changes

pradyunsg approved these changes Jul 17, 2022

View reviewed changes

sbidoul merged commit a7a35dc into pypa:main Jul 18, 2022

sbidoul deleted the pip-inspect-sbi branch July 18, 2022 06:10

github-actions bot locked as resolved and limited conversation to collaborators Aug 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pip inspect command #11245

Add pip inspect command #11245

sbidoul commented Jul 10, 2022 •

edited

Loading

pradyunsg commented Jul 10, 2022

sbidoul commented Jul 12, 2022

pradyunsg commented Jul 13, 2022

pradyunsg commented Jul 13, 2022

sbidoul commented Jul 13, 2022

uranusjr Jul 15, 2022

sbidoul Jul 15, 2022

pradyunsg Jul 15, 2022

Add pip inspect command #11245

Add pip inspect command #11245

Conversation

sbidoul commented Jul 10, 2022 • edited Loading

pradyunsg commented Jul 10, 2022

sbidoul commented Jul 12, 2022

pradyunsg commented Jul 13, 2022

pradyunsg commented Jul 13, 2022

sbidoul commented Jul 13, 2022

uranusjr Jul 15, 2022

Choose a reason for hiding this comment

sbidoul Jul 15, 2022

Choose a reason for hiding this comment

pradyunsg Jul 15, 2022

Choose a reason for hiding this comment

sbidoul commented Jul 10, 2022 •

edited

Loading