Custom files under dist-info as part of a build hook

[cadojo]

I'm working on a package manager project that will add a unique identifier to each package. The [project] metadata is standardized, as are the contents under the METADATA file in dist-info/. I would like to include the UUID of each package under a new file called dist-info/UUID, or similar.

Can this be done using a Hatchlings build hook? Reading through the documentation, I'm not sure if "metadata" refers to the pyproject.toml metadata, or the metadata as compiled by the build tool, and written to dist-info.

[ofek]

extra-metadata https://hatch.pypa.io/latest/plugins/builder/wheel/#options

[cadojo]

Is the "mapping" referenced in the documentation "pyproject.toml key" to "filename under extra_metadata? For example, is this the intended usage?

[project]
...

[tool.my_tool.metadata]
uuid = "1234"

[tool.hatch.build.targets.wheel.extra_metadata]
"tool.my_tool.metadata.uuid" = "UUID"

[ofek]

[tool.hatch.build.targets.wheel.extra-metadata]
"./UUID.txt" = "UUID"

Your hook would need to write to the source location e.g. ./UUID.txt. If that is only known at build time then I'll need to add support to https://hatch.pypa.io/latest/plugins/builder/wheel/#build-data

[ofek]

or just commit a file...

[ofek]

You're the first to use this feature btw, hooray for premature optimization 😄

[cadojo]

Well first off thanks for the quick replies!

I do want the data to live in the pyproject.toml file, not in a committed or external file. I posted on discuss.python.org about adding custom metadata to the ***.dist-info/METADATA file. The specific project I'm working on — a toy package manager that is registry-focused, as opposed to index-focused — would ideally read metadata for installed packages from the standard .dist-info directory. See the discussion below for motivation for the "feature" or "functionality" I'm talking about in this issue.

https://discuss.python.org/t/add-custom-metadata-to-wheel-file/21410/2

No need to add support to build-data, if this is possible with a custom hook I'm happy to write my own and release it as a plugin!

[ofek]

Thanks for the context!

I do want the data to live in the pyproject.toml file, not in a committed or external file [...] No need to add support to build-data

Since the option reads the sources from disk this implies your hook would write to a temporary file which is only known at build time which means I do need to add support to build-data

[cadojo]

Hm okay, I believe you but I do not understand why more changes are necessary — of course, my understanding of Python packaging has a ton of room to grow, so that's not surprising.

I can't do this because the [project] key is standardized, but what I want to do is add a field under the [project] key called uuid, right alongside the package name. Instead of putting uuid in the standard [project] key, I'll put it under [tool.my_tool.project] or something.

Which step in this process requires the changes to build-data? Some data is pulled from pyproject.toml and stuck into dist-info already, like the package name and description. Why can't this new field, uuid, be written to a file under dist-info in the same way?

[ofek]

To illustrate, please:

1. Create a ./UUID.txt
2. Add:

   [tool.hatch.build.targets.wheel.extra-metadata]
   "./UUID.txt" = "UUID"

3. Run hatch build
4. Look inside the dist/*.whl ZIP file
5. Finally, imagine how that would work if there was no existing file

[cadojo]

I've done so, and I think I understand. Right now, extra-metadata provides the ability to copy <FILE> to dist-info/extra_metadata/<FILE>. And I'm asking for copying information from pyproject.toml to dist-info/extra_metadata/<NEW_FILE>. Is that correct?

If so, I didn't post this issue to create work for you. I understand if this is a feature hatch will not support, though I'm happy to see your Discourse post.

➜ tree python_pkg-0.1.0.dev0.dist-info 
python_pkg-0.1.0.dev0.dist-info
├── METADATA
├── RECORD
├── WHEEL
├── extra_metadata
│   └── UUID
└── licenses
    └── LICENSE

➜ cat python_pkg-0.1.0.dev0.dist-info/extra_metadata/UUID 
855f5fd5-12cd-4c2c-9cb7-ceb657731c58

[ofek]

Right now, extra-metadata provides the ability to copy <FILE> to dist-info/extra_metadata/<FILE>. And I'm asking for copying information from pyproject.toml to dist-info/extra_metadata/<NEW_FILE>. Is that correct?

Correct

I understand if this is a feature hatch will not support

I will, gimme a few hours

[ofek]

#621

No promises on a release timeline

[cadojo]

Thank you!!!

[ofek]

Anytime!

[pradyunsg]

FWIW, the dist-info directory is supposed to be a flat set of files that is supposed to be modelled as a dictionary of "name": "text content" (at least, based off of my understanding from chats during the installer's API design discussions).

[ofek]

Perhaps we can standardize a user-defined extra_metadata directory

[pradyunsg]

Perhaps -- but right now, that violates the designers' expectations for how files in dist-info are handled. I don't think any other backend allows injecting files in there directly, but it's probably worth disallowing folders to be cautious IMO. :)

[ofek]

Directories already exist inside there as of PEP 639 with licenses/

[cadojo]

I'd also note that the use case I want (that motivates this issue) is just one more "name": "text" file. It could go anywhere, including top-level .dist-info, but I do think it makes sense to have non-standard files under an "extras" directory.

[cadojo]

If directories are truly disallowed, perhaps user files could be placed at the top-level. Though as @ofek notes, the licenses directory already exists.

[pradyunsg]

I will remind folks that PEP 639 is still in draft state.

[ofek]

Sure, but nobody has expressed any disagreement with a dedicated directory for licenses. If you personally do then you might want to bring it up on Discourse

[pradyunsg]

Oh, I don't have a strong opinion but it's a thing we're changing and the design needs to consider/document the potential implications of that. I've already mentioned this, when I wrote the above comment, over on the discussion about designing the draft PEP: https://discuss.python.org/t/pep-639-round-2-improving-license-clarity-with-better-package-metadata/12622/86?u=pradyunsg