`dune subst` inferred version is totally misleading and wrong

[toots]

We were debugging several user builds of our code base and were faced with a spread of totally confusing versions returned from building from a git checkout.

Turns out that dune subst uses the following command to infer dev versions:

git describe --always --dirty --abbrev=7

But this command produces descriptions that are absolutely misleading. Here's an example:

% git describe --always --dirty --abbrev=7 --debug                                                                                                                   
describe HEAD
No exact match on refs or tags, searching to describe
finished search at 6eba12641e468c435114b2afc8763211f41b168a
 annotated        888 v2.2.0
traversed 889 commits
v2.2.0-888-g748cd0c

This is using a commit that is 889 commits (!!) behind the current main (!) branch to describe it and using the versioned tag, which would have you wrongly assume that this is a build for that version.

Also that g prefix before the commit is really misleading as well.

Lastly, tags are by definition moving pieces in git repo. Typically, a release tag can be force-pushed and etc. They are not very useful to have a reliable version. A commit SHA would be much better.

This command should be tightened to return something closer to the current commit. I would expect that most people simply want the current branch name, if it exist, and/or the commit, something like:

[<branch-name>|<tag name>-]-git@<commit SHA>

At the very least, the command should be updated to:

• Include all tags (some git clones don't have them all locally checked out)
• Limit the number of commit to look for. I'd frankly suggest just one!

This gives:

git describe --always --dirty --abbrev=7 --candidates=1 --tags

[maiste]

Thanks for reporting!
@rgrinberg, @emillon is there any blocker that would prevent this change? I would expect people to set the version when they need to. As a result, I don't see a problem for dune to do the substitution this way when no version is provided?

[MisterDA]

This is a difficult problem. It happens often that tags are not part of the main branch, and that the most recent tag reachable from the last commit is quite old. If I look at ocaml's trunk branch for instance:

$ git describe --long --abbrev=7 | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
4.12.0.alpha1.r11226.gc5cde12

I'd suggest looking at Arch Linux VCS package guidelines for the pkgver function. I think that as long as the string Dune reports contains the commit revision, the tag can be allowed to be a bit unreliable.

Meson suggests using an external script: Cement a version obtained from VCS. It also has a vcs_tag function.
Autoconf has git-version-gen.

[toots]

Just wanted to send a ping on that again. All our bug reports from users using a recent version are wholly unusable when it comes to the version they run. I think we might have to re-introduce our own custom versioning hack.

See for instance: savonet/liquidsoap#4794. Version says: v2.2.0-1314-gdb59203+dev but the user is in fact on version 2.4.1+dev. It's not just wrong, this will give the user the impression that they are on a totally wrong version branch.

[rgrinberg]

The way we get around this in dune is always merging tags back to main.

With that being said, your proposals are fine as well. With that being said, this needs to be done in a way that preserves behavior for users aren't upgrading their project version. If you'd like to fix this, I'd suggest to add a flag to dune subst that has the behavior that you want, and then modifying dune to generate this new invocation if some sort of dune-project is set.