Git tags - support for multiple versions on same commit

[andrewmichaelsmith]

I think that the current mechanism for getting tags (improved in #77) doesn't pick the biggest version if you tag the same commit multiple times for the same application but for different versions.

Basically when I bump a version number for my application without doing a commit (not that common but does happen if we update a package's dependencies) the older tag is getting picked up.

Probably easiest to illustrate just using git and the current "git describe" command:

# git init .
Initialised empty Git repository in /tmp/x/.git/
# touch x && git add x && git commit -m "test"
[master (root-commit) 217feb2] test
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 x
# git tag R_example_1.0.0
# git tag R_example_1.0.1
# git describe --tags --dirty --always --long --match "R_example*"
R_example_1.0.0-0-g217feb2

Here 1.0.0 is getting taken over 1.0.1, clearly I'd like to have 1.0.1 taken over it.

From initial googling there seems to be a way to get the current tags for a commit. I suppose you could take these all in, parse them and pick the biggest? But I guess this doesn't give you the nice things that describe does (like dirty and commits ahead).

[warner]

Hm, one other (perhaps more common) way this might happen is if a release candidate gets promoted to the final release without additional changes, so you'd have "v1.0rc1" and "v1.0" tags on the same commit. (This is probably the ideal outcome in any process that involves "release candidates": making no changes between your last candidate and the final release). In that case, however, the more-final less-candidate-ish tag is shorter, so "shortest tag wins" still works.

How do you update the package dependencies without changing the source code at all? I guess the dependencies are being tracked externally somehow?

I'm thinking we could include a list of all tags in the get_versions() dictionary output. The Versioneer code could have some built-in sorting function to decide which one makes it into the pre-rendered version string, but project-specific logic could look at the full tag set and do its own sort to decide which one to use. Yeah, like you said, not so easy to integrate with the other version information like "distance". I guess the consequence would be that revisions after a multiply-tagged commit would have non-ideal version strings, maybe 1.0.0+4.gHASH where it should have been 1.0.1+4.gHASH.

The built-in sort would ideally use the PEP440 version-sort logic (so it'd pick 1.0.1 over 1.0.0), but I don't want to have to import an external library to do the sort, so I'm not sure of the best way to bring that code into versioneer.py .

Maybe it'd be good enough to:

1. find all tags with the given prefix, strip the prefix
2. find the length of the shortest tag, discard all tags longer than this
3. sort the remaining (short) tags lexicographically, pick the "largest"

That'd choose 1.0.1 over 1.0.0, but also 1.2.13 over 1.10.2 . We could assume dotted-decimal and sort(key=lambda tag: tuple([int(c) for c in tag.split(".")])), which I think is what Debian does, but it feels kinda dicey to make that assumption (I'd probably write it to only attempt that if all the tags matched a [\d\.] regex, and fall back to straight string-sort otherwise).

[andrewmichaelsmith]

"Updating dependencies without updating code": We may have a 1st party dependency pinned at only the major (library>=1,<2). We trust our internal changes to not make breaking changes (and don't want to have to update minor versions each time a minor library changes). Then we'll rebuild our application (run our tests, etc.) with that library and "release" that as a new version.

Yes, your RC example is probably a better example of this use case.

I notice you use setuptools in some places, is this a dependency you can rely on being there? setuptools' parse_version is PEP 440 compliant and looks to do what we need: https://pythonhosted.org/setuptools/pkg_resources.html#parsing-utilities

Edit: I see you depend on "setuptools" or "distutils", so I think it's possible to use setuptools' parse_version and (failing that) distutils' StrictVersion (http://nullege.com/codes/search/distutils.version.StrictVersion)

[warner]

Ah, now I get it: the compiled executable is your versioned artifact, rather than your source tree. Compiling the same source tree at two different times (between which a new upstream dependency was released) results in two different executables, and you want those executables to have different version strings.

Yeah, I guess in all the projects I've managed, we do that by having the CLI --version command print out the version of all the dependencies, as well as the top-most package version, and treat the version string as sort of being multidimensional. The top-most package version describes only its own source code. But I can see how that's not very easy to use: when folks file bug reports against your executable, you want a single short string, not a multi-line list of versions for every dependency you've got. I suppose you might try building a post-source-tree version string which concatenates the top-level package version with a hash of a serialized list of all the subdependency versions, and get stuff like "1.1.0-depHASH", which could be looked up in a table when the bug report comes in. Or you could use an automated bug-reporting tool which includes all the subdep versions in the machine-generated report. Or.. hm, maybe build the executable version string out of the Versioneer string plus a single number that you pass in during the build process: then you put one tag per commit on your source tree, and each rebuild you increment the extra number. That extra number is referring to something outside the source tree anyways, so recording it inside the source tree doesn't accomplish much (you couldn't rebuild the same executable from just the version string.. you need a table somewhere that records the exact upstream versions that went into the build). You could even pretend that the extra number was a part of the main version: use a tag of v1.0, and a build-time number of "3", to produce v1.0.3.

Anyways, yeah, we could try to import parse_version, from setuptools, although one problem with import setuptools is that it monkeypatches distutils the moment it gets imported, and I kinda think Versioneer shouldn't force people to use setuptools by accident. I guess we'd either want to use distutils.StrictVersion all the time, or only use setuptools's if "setuptools" in sys.modules already, to avoid accidentally being the ones to import it for the first time.

[SvenRtbg]

Tagging with annotated tags not only adds the timestamp and committer account to this tag (and a message that probably nobody cares about), but also allows Git to pick the more recent tag, i.e. the one added later in time, when running git describe.

[uda]

git describe does not return the latest tag, just the first (by alphabetic order) tag for the relevant commit, This does by date: git tag --sort=-creatordate --points-at HEAD 'v*', but it only returns the tag.

[SvenRtbg]

You may be referring to lightweight tags - that's why I explicitly talked about annotated tags:

$ git init
$ touch file
$ git add file
$ git commit -m "init"
[master (root-commit) d2f0c94] init
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 file
$ git tag -am "first" "x"
$ git describe
x
$ git tag -am "second" "a"
$ git describe
a
$ git tag -am "third" "z"
$ git describe
z

Following your logic, the last line should have returned "a" as the tag. It returns "z" because that is the last tag created on the commit, ordered by timestamps of the tags.

$ git tag
a
x
z
$ git tag --sort=-creatordate
z
a
x

$ git --version
git version 2.21.0.windows.1

[uda]

OK, thanks, this is actually a great workaround for our current issues and a great practice anyway...

[SvenRtbg]

git describe will complain if it does not find annotated tags in the repository. And then when made to work with lightweight tags will behave exactly as you described and return the alphabetically lowest tag.

[jeichel-miovision]

For my purposes I added a block of code after pieces["closest-tag"] = ... to get the most recent tag

...
        pieces["closest-tag"] = full_tag[len(tag_prefix) :]

        # try to find most recent from multiple tags
        points_at_out, rc = run_command(
            GITS, ["tag", "--sort", "committerdate", "--points-at", "%s%s" % (tag_prefix, pieces["closest-tag"])],
            cwd=root
        )
        # --points-at was added in git-2.4.0
        if points_at_out is not None:
            points_at_out = points_at_out.strip().split('\n')
            for version in points_at_out:
                if version.startswith(tag_prefix):
                    pieces["closest-tag"] = version[len(tag_prefix) :]

        # distance: number of commits since tag
        pieces["distance"] = int(mo.group(2))
...