Add support for dynamic metadata

[Czaki]

In my project I use setuptools_scm for versioning (it calculate version from git tag and create proper file handling it). My project is distributed as python package and pyinstaler executable. I would like to switch from pyinstaller to briefcase (possibility of pip support) but I do not want to manually change version in file.

Describe the solution you'd like
Add support to read version from setuptools_scm instead of manual update

Describe alternatives you've considered
Manual updating version. read version from file

[freakboy3742]

Thanks for the suggestion. I agree that "version from SCM" is a use case worth considering.

There are a couple of complications to what you describe, though. Some packaging schemes have very strict version number requirements. Windows only allows strict numerical Triples. Android requires a build "sequence" number. IIRC, iOS only allows numbers. As a result, building an installer "for git hash xxxxx" isn't actually something you're going to be able to do in practice; even "for git tag xxxxx" will have some very serious constraints. We'll need to work out how to manage those constraints in any SCM-based version number solution.

There are also alternatives to setuptools_scm out there (versioneer being one that I am aware of); we'll need to make a decision whether to make an opinionated choice of one of those packages, or different version control systems (or even introduce a plugin system to support alternatives).

[Czaki]

So maybe add option to provide version by function?
Something like get_version.py:get_version Then user could provide simple wrapper function which provide version in proper format?

[freakboy3742]

A user-provided function would be one way to provide compliant version numbers; however, that function would have to either (a) return a PEP508-compliant version number, which is then processed using the existing version number handling code, or (b) return a different version number for each platform.

What we really need at this point is a complete proposal for needs to change, and what the experience will be like as an end-user.

[Czaki]

A user-provided function would be one way to provide compliant version numbers; however, that function would have to either (a) return a PEP508-compliant version number, which is then processed using the existing version number handling code, or (b) return a different version number for each platform.

But is is enough to add description in documentation for this and fail if user function return version in wrong format.

[mhsmith]

As mentioned in #1277, there is now a standard dynamic setting in pyproject.toml which may be useful for this.

[andrewleech]

The official dynamic version support in pyproject.toml is handled deep within the setuptools / disutils / Distribution system. I've attempted to trace through how this works to find a simple means of querying it within briefcase, however it doesn't appear to be exposed in any official api as such.

Essentially:

• [project] needs to include dynamic = ["version"]
• the [build-system] section needs to have the dynamic version providing package listed in its requires
• the dynamic version providing package needs to be enabled with a [tool.<versioning package>] block

Once these are done, part of setuptools.dist.Distributrion will call through to setuptools.finalize_distribution_options registered entry-point in the versioning package which then figures out the real version and sets it on the Distribution object.

However, there's an alternate format of configuring dynamic version without needing a package that supports it, by simply setting a block in pyproject.toml:

[tool.setuptools.dynamic]
version = {attr = "__version__.version_py_short"}

This one gets flagged with a warning from setuptools that it's still beta functionality.
If both mechanisms are configured, I'm not sure which one wins.

Basically... if briefcase simply built the app as a wheel or sdist first with the standard tooling (eg. the build tool) and then extracted said wheel, then we'd get all the dynamic versioning stuff for free using the official mechanisms.

Trying to replicate the official functionality might be possible, but probably a lot of busywork and I'd also expect it to be somewhat fragile.

[layday]

I know the Python packaging landscape is (more than) a little confusing, but there's nothing I'd call "official" dynamic version support in pyproject.toml. Declaring version as dynamic has no significance beyond informing the reader that the version is defined elsewhere. setuptools does not "own" project configuration, and other build systems might handle extracting the version from VCS or code differently (or not at all) - see e.g. Flit, poetry-dynamic-versioning (based on dunamai, which is what I'd use if briefcase were to expose a version hook), hatch-vcs.

[andrewleech]

I guess I meant official in the sense of how the current pypa backed tooling uses pyproject.toml to build sdist/wheels.
There's certainly no official version provider package (though setuptools-scm is used as the reference example) but this, my preferred git-versioner and all the ones you list that directly support pyproject.toml do so you via "build-system" integration, regardless of whether that build system is setuptools or their own builder.

Briefcase should really be able to work with any dynamic version provider if they all follow a standard pattern... though most of the standard patterns are discounted l designed to ultimately build a wheel.

[andrewleech]

That being said, the setuptools workaround pattern could be reused here, providing a section like this:

[tool.briefcase.dynamic]
version = {attr = "__version__.version_py_short

Which allows the user to define a property/function to call to provide the version.

[freakboy3742]

Briefcase should really be able to work with any dynamic version provider if they all follow a standard pattern... though most of the standard patterns are discounted l designed to ultimately build a wheel.

So - I agree in theory - but I'm not sure there is any such thing as a standalone dynamic version provider in the way you're describing it. All the tools I'm aware of exist as plugins to other build systems (setuptools, hatch, poetry etc). It may be possible to adopt one of these tools, but the configuration of that tool is going to be something that is Briefcase specific.

I'm also not sure I understand why the ability to configure multiple SCM version number utilities is required. Yes, the ecosystem has a proliferation of near-identical tools... but AFAICT this is mostly due to no single tool gaining traction due to the general mess of the broader Python packaging ecosystem, rather than a demonstration of the highly customised needs of individual versioning schemes.

With that in mind, I make the following suggestion:

If a project only defines version, whether that be in the [project] PEP621 section, or a [tool.briefcase.*] section, it's a hard-coded version number. This is the status quo; we already do validation that the provided string is PEP440 compliant.

If a project defines a PEP621-compliant dynamic = ["version"], version in [tool.briefcase.*] is allowed to be:

• {"file": "path/to/file"}, will grep the contents of the named file, relative to pyproject.toml, looking for __version__ = "(.*)", and use the value match (returning an error if the file can't be found, or the regex doesn't match. PEP440 validation is done on this string.
• {"attr": "myapp.module.__version__"} will do importlib calls to import "myapp.module" and retrieve the __version__ attribute, using the same PYTHONPATH used to run briefcase dev. PEP440 validation is also done on this string.

If version is undefined at the briefcase level, setuptools_scm is used to extract the version number, using the path containing pyproject.toml as a root.

The decision to use setuptools_scm is based entirely on the fact that Briefcase itself uses setuptools_scm, and it's the implementation that is used by a lot of the other build tool plugins (e.g., hatch-vcs is really just a wrapper around setuptools_scm). If someone wants to make a particularly impassioned plea to use a different library, I'm open to reasonable discussions, but this would probably need to be paired with Briefcase (and other BeeWare projects) making the same choice.

The decision to make setuptools_scm the default if version is undefined is that "discover what you can from the VCS" seems like a reasonable default fallback position, and it requires no additional configuration.

In theory, we could expose a plugin interface - allow anyone to define a briefcase-myversionplugin tool that registers their own versioning scheme... but I wouldn't consider that necessary for a v1. I'd rather deploy a 95% solution and see if there's demand for the last 5%.

[layday]

I'm also not sure I understand why the ability to configure multiple SCM version number utilities is required. Yes, the ecosystem has a proliferation of near-identical tools... [...]

There are very few SCM version number utilities (I can only think of two: setuptools_scm and dunamai). What there is a preponderence of is build system plug-ins. There's no unified build system plug-in interface (although there is a proposal for one), so people have got to write separate plug-ins for each build system which supports dynamic metadata. setuptools_scm is both an SCM version provider and a setuptools plug-in. hatch-vcs uses setuptools_scm under the hood to extract SCM versions and I assume other build systems do too.

If a project defines a PEP621-compliant dynamic = ["version"], version in [tool.briefcase.*] is allowed to be: [...]

Just to note, and assuming these are inspired by setuptools, the file attribute in setuptools simply loads the contents of a text file. attr AST-parses the module to avoid import side-effects (importing other modules).

[freakboy3742]

I'm also not sure I understand why the ability to configure multiple SCM version number utilities is required. Yes, the ecosystem has a proliferation of near-identical tools... [...]

There are very few SCM version number utilities (I can only think of two: setuptools_scm and dunamai). What there is a preponderence of is build system plug-ins.

That would seem to reduce the impetus for a Briefcase plugin even further.

Just to note, and assuming these are inspired by setuptools, the file attribute in setuptools simply loads the contents of a text file. attr AST-parses the module to avoid import side-effects (importing other modules).

"Inspired by setuptools" was the intention, so I agree we should follow those implementations.

I know the attr parsing was the subject of an old issue, but wasn't aware it had been resolved. Either way, I agree a parsing-based, side-effect free implementation is preferable.

[mhsmith]

Renamed to reflect other possible types and sources of metadata, such as this example from @DaloroAT in #1432:

---

I have my own project where I'm using Dynamic Metadata for the version field. pyproject.toml looks like

[project]
name = "my-project"
dynamic = ["version"]

[tool.setuptools.dynamic]
version = {file = "my-project/VERSION"}

I have the file VERSION in the source folder of my-project with a version value as a plain string. During package installation, this version was substituted.