Installation/resolution report (aka pip install --dry-run --report)

[sbidoul]

Following the conversation in #10748 and @cosmicexplorer 's request that I describe more precisely what I had in mind I created this proof-of-concept.

The output is conceptually very similar to #10748, but generated as part of the install command.

It enables the following use cases:

1. Tell me what pip installed: pip install --report
2. Tell me what pip would install: pip install --dry-run --report
3. Tell me what pip would install in an empty environment (aka resolve the requirements): pip install --dry-run --ignore-installed --report

It is not exactly the same as I take the opportunity to experiment a little bit on the output format.

My goal here is to have a standards based output, using PEP 610 for download_info and PEP 566 for metadata.

So far I did not need to intrude into the resolver innards as I can obtain all the information from InstallRequirement, by the way of a minor change to the preparer to record the downloaded artifact.

Missing so far, compared to #10748, is dependencies and required python information, but this will be part of the metadata field obtained form the distribution metadata. TODO about metadata is convert it to PEP 566 json.
Dependencies and required python information are part of the metadata fields in PEP 566 json format.

Here is an example:

$ pip install "pydantic>=1.9" git+https://github.com/pypa/packaging@main --report=/tmp/report.json --ignore-installed --dry-run --no-cache

{
  "version": "0",
  "pip_version": "22.2.dev0",
  "install": [
    {
      "is_direct": false,
      "download_info": {
        "url": "https://files.pythonhosted.org/packages/78/b0/fdedac2f07344035607bfbf42217c103a6421e7845fc3cb9fd07f3fa0d2e/pydantic-1.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
        "archive_info": {
          "hash": "sha256=574936363cd4b9eed8acdd6b80d0143162f2eb654d96cb3a8ee91d3e64bf4cf9"
        }
      },
      "metadata": {
        "name": "pydantic",
        "version": "1.9.0"
        ...
      },
      "requested": true
    },
    {
      "is_direct": true,
      "download_info": {
        "url": "https://github.com/pypa/packaging",
        "vcs_info": {
          "vcs": "git",
          "requested_revision": "main",
          "commit_id": "ba124324e866e518ebfe0378a25b6ba4816e5880"
        }
      },
      "metadata": {
        "name": "packaging",
        "version": "21.4.dev0"
        "requires_dist": [
          "pyparsing (!=3.0.5,>=2.0.2)"
        ],
        "requires_python": ">=3.7",
        ...
      },
      "requested": true
    },
    {
      "is_direct": false,
      "download_info": {
        "url": "https://files.pythonhosted.org/packages/d9/41/d9cfb4410589805cd787f8a82cddd13142d9bf7449d12adf2d05a4a7d633/pyparsing-3.0.8-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=ef7b523f6356f763771559412c0d7134753f037822dad1b16945b7b846f7ad06"
        }
      },
      "metadata": {
        "name": "pyparsing",
        "version": "3.0.8"
        ...
      }
    },
    {
      "is_direct": false,
      "download_info": {
        "url": "https://files.pythonhosted.org/packages/75/e1/932e06004039dd670c9d5e1df0cd606bf46e29a28e65d5bb28e894ea29c9/typing_extensions-4.2.0-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=6657594ee297170d19f67d55c05852a874e7eb634f4f753dbd667855e07c1708"
        }
      },
      "metadata": {
        "name": "typing-extensions",
        "version": "4.2.0"
        ...
      }
    }
  ],
  "environment": {
    "implementation_name": "cpython",
    "implementation_version": "3.8.10",
    "os_name": "posix",
    "platform_machine": "x86_64",
    "platform_release": "...",
    "platform_system": "Linux",
    "platform_version": "...",
    "python_full_version": "3.8.10",
    "platform_python_implementation": "CPython",
    "python_version": "3.8",
    "sys_platform": "linux"
  }
}

Please let me know your thoughts and if I'm missing anything.

Closes #53
Closes #6430

[cosmicexplorer]

Note that in a67c59f I have refactored #10748 to make its output easier to consume from a pip install command.

[sbidoul]

I have updated this PR to produce hashes in more situations. The only missing part is the integration with the wheel cache (on which I started working in #11042).

All in all, this seems to fit pretty neatly in the pip code base.

The hardest remaining part is to produce PEP 566 compliant metadata. To achieve this we may need to vendor importlib_metadata which has the metadata json conversion function. On the other hand, for a MVP it could be sufficient to export limited metadata information such as Name, Version and maybe Requires-Dist.

Also --dry-run needs to print/log something, but that should be easy.

If there is positive feedback in the short term (especially from people who want to produce lock files with hashes) I'm motivated to push this forward. I may be able free up some time in May so as to land this as an experimental feature in 22.2 (July).

[pfmoore]

The hardest remaining part is to produce PEP 566 compliant metadata. To achieve this we may need to vendor importlib_metadata which has the metadata json conversion function.

I have an implementation that converts a METADATA file (more accurately, an email.message.Message object) to that format in https://github.com/pfmoore/pkg_metadata/blob/main/src/pkg_metadata/metadata.py (I'd recommend extracting that function, rather than vendoring pkg_metadata at this point, as I only just uploaded the first draft version of the project yesterday!). The msg_to_json function is pretty robust, though - it successfully translates the METADATA file for every wheel on PyPI (including some that have things like broken encodings).

[sbidoul]

Thanks @pfmoore! I extracted your msg_to_json and it fits nicely.

The PR is still pretty small but it will grow with tests and docs etc. So my plan is to split it in easier to review chunks

• Add --dry-run option to pip install #11096
• Add json_metadata property to BaseDistribution #11095
• Refactor BaseDistribution.metadata property to make sure it has Requires-Dist and Provides-Extra #11163
• side quest: Add metadata field to pip list json output #11097
• adding download_info: DirectUrl on InstallRequirement and integrating with the wheel cache: Add download_info: DirectUrl to InstallRequirement #11137
• pip install --report (this PR)

In the meantime, this branch is ready for experimenters, and I'm happy to discuss / bikeshed the report schema here.

[cosmicexplorer]

I am about to propose basing #10748 off of this PR, since I really like the formalization of metadata parsing in _internal/metadata/json.py and moving the InstallationReport itself into _internal/models.

I had written a whole treatise trying to convince you to move this to pip download --report, but after reading #10748 (comment) again, it makes a lot more sense to me, and I now agree with you on keeping it in pip install --report. Would you consider:

1. renaming --ignore-installed to --empty-env?
2. renaming InstallationReport to ResolutionReport?

And as i am about to propose in #10748, I think I can make that PR additionally avoid downloading any dists when --dry-run is provided.

EDIT: see proposal at #10748 (comment) (which doesn't change anything here, just refactors that PR to consume this one).

[sbidoul]

renaming --ignore-installed to --empty-env?

@cosmicexplorer actually the --ignore-installed option already exists. So it would mean creating an alias. Perhaps this should be discussed in a separate issue ?

renaming InstallationReport to ResolutionReport?

I'll look into it. This could make sense if we use it in another context than the install command.
OTOH, this name does not appear in the JSON output so it is easy to change as it is part of the pip internals.

[sbidoul]

I rebased on top of the other PRs and added integration tests. This is now feature complete.

I keep it draft until the other PRs are merged but this PR (i.e. the last commit) is now simple as it can get.

[sbidoul]

Looks good to me, simple and straightforward.

Thanks @uranusjr!

Also, the groundwork that was required to make this PR simple will be useful for other things.

[sbidoul]

I have made a last tweak to this one. Instead of a dict keyed by canonical distribution name, the install field is now an array.
I feel the dictionary was not bringing any significant benefit to users, since nothing else in the data referred to these dictionary keys. The dictionary was also possibly restrictive (such as losing installation order information).

[sbidoul]

I plan to merge this in about a week so it goes in 22.2, if there are no other comments or concerns in the meantime.

[sbidoul]

Thanks a lot to all who contributed to this!

[woodruffw]

Just wanted to say as a downstream user: thanks a ton for adding this!