Metadata 1.3 PEP

[ncoghlan]

Internal PRs within a fork are a handy format for reviews prior to submitting a PR against the main repo :)

[ncoghlan]

I like this as a starting point. I've made a few PEP editing comments inline, and added a comment noting the other changes I'm aware of that need to be backfilled.

[ncoghlan]

distutils-sig <distutils-sig at python.org>

[ncoghlan]

Initial state should be Draft

[ncoghlan]

Replaces: 345

[ncoghlan]

Additional changes to cover:

• Name format is defined in https://www.python.org/dev/peps/pep-0508/#names
• new field Description-Content-Type
• new field Provides-Extra

[di]

Could you explain why you think this PEP should explicitly include the two new fields? I opted not to because I felt that it somewhat defeats the purpose of this PEP (i.e. that new fields are now specified elsewhere, not in this PEP.)

Do you think this PEP should include the entire specification for the two new fields, or just mention here (in the "Summary of Differences" section) that they have been added?

In your mind, if we needed to add another field after the acceptance of this PEP, what would happen? Would we amend this PEP? Or would that field be part of "Metadata 1.4"? Would a new metadata version require a new PEP?

Along those lines, should this PEP mention something about when and why the metadata version number should change?

[ncoghlan]

The goal of the process change is to switch to a model where distutils-sig PEPs describe changes to the specifications, but are not themselves used as primary reference documents. The big difference that makes is that it means we don't copy and paste unchanged fields into the new PEP - we only list the new & updated ones.

Since this is a backfill PEP, you're creating a record of past decisions already made in GitHub PRs rather than actually requesting approval of the changes, but in any future metadata update PEPs will also need to make the argument for why particular additions or changes are needed.

For moving the reference URLs to PyPUG, the appropriate reference would be: pypa/pypa.io#11

For Description-Content-Type the appropriate reference would be: pypa/packaging.python.org#258

For Provides-Extra, the most appropriate reference would be probably be to an early draft of PEP 426, such as: https://github.com/python/peps/blob/493ff3b7b5e6f83e7ebaf6ac8902fcdc8835abbc/pep-0426.txt#L428

[ncoghlan]

A shorter version of that comment: rather than being about what the field is, the entry in the PEP should focus on why we added it.

Provides-Extra: so installers know which extras are available for selection, and so publishing tools can check for typos in environment markers.
Description-Content-Type: so tools (most notably PyPI) can be told how to render the long description, rather than always assuming the use of reStructuredText

[ncoghlan]

I know you said you'd like to keep this as simple as possible, but I think it will be worth our while to define a canonical transformation from the current line-oriented email-header style formatting (which requires reading the PEP in with email.parser.HeaderParser and accounting for multiple use fields yourself) and a JSON-compatible data structure.

As a first draft of that:

1. The existing key-value format should be read with email.parser.HeaderParser
2. All keys should be reduced to lower case, but otherwise left alone
3. Multiple-use fields should be collected into a single list value for that key
4. Keywords should be converted to a list by splitting on whitespace
5. The result should be stored as a string-keyed dictionary

[di]

Thanks very much for the comments, @ncoghlan. I just have a handful of questions and then I should have another pass at this for you.

[di]

@ncoghlan This is ready for another review.

[ncoghlan]

This version looks pretty good to me as a starting point for distutils-sig discussion.

If you turn it into a PEP 566 PR against the main PEPs repo, then I can merge it, and you can start the thread on distutils-sig (that should hopefully be short, but folks may spot holes in the proposed JSON conversion algorithm)

[ncoghlan]

This opening paragraph should change. Something like:

This PEP describes the changes between versions 1.2 and 1.3 of the core metadata specification for Python packages. It also changes to the canonical source for field specifications to the Core Metadata Specification_ reference page on packaging.python.org.

And then adjust the rest of the abstract accordingly (e.g. while mentioning PEP 345 is good, I don't think you need to mention the others)

[ncoghlan]

Grammar here isn't quite right. Perhaps "... to store metadata in a ..."?