Could pandoc --version indicates a different version for nightly builds of current unreleased Pandoc ?

[cderv]

Pandoc is built daily through Github action (https://github.com/jgm/pandoc/actions/workflows/nightly.yml) and this is really useful to be able to tests our external toolings based on next Pandoc version. For example, we use these builds in CI tests to detect breaking change before a new Pandoc version is released. This complete our tests suites of several Pandoc versions.

One thing that is not easy to overcome is that nightly version of Pandoc does have the same version number as the last Pandoc release.

pandoc.exe 2.18
Compiled with pandoc-types 1.22.2, texmath 0.12.5, skylighting 0.12.3,
citeproc 0.7, ipynb 0.2, hslua 2.2.0
Scripting engine: Lua 5.4
User data directory: C:\Users\chris\AppData\Roaming\pandoc
Copyright (C) 2006-2022 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

This means there is no easy way to know that we are running a newer Pandoc version than last released one. However, the newer version could already have changes we need to deal with and usually adding conditional testing as we want support also for older version.
It would be good to have a way to know from Pandoc CLI itself that we are not running last release (2.18 here) but a newer version (2.18+). Then we could just add a condition like

if (pandoc version > 2.18) then 
  test new behavior
else 
  test old behavior

This can't be done for now as pandoc --version would give the same version number.

About potential solution,

I think that one could be in the version scheme itself which is common for software I believe. Pandoc follows Haskell versionning policy and already uses up to 4 components. This would probably mean adding a 5th one. Devel version would be on 5 numbers (2.18.0.0.0 = devel after 2.18, 2.14.0.3.0 = devel after 2.14.0.3, ...) with the last number incremented at each nightly build or not incremented to simplify and staying with 0, or maybe another one like 9 or 99 to be more clear on devel status (2.18.0.0.99 for devel right after 2.18, or 2.14.0.3.99 for devel after 2.14.0.3).

If version scheme change is not an option, then it could simply be an indication in results from pandoc --version that could be parsed.
This is less ideal because it means having specific parsing step to detect development version, and comparing version number would not show that the nightly build is above the last release one.

This could be something like

pandoc.exe 2.18
+ nightly build from commit 810879a
Compiled with pandoc-types 1.22.2, texmath 0.12.5, skylighting 0.12.3,
citeproc 0.7, ipynb 0.2, hslua 2.2.0
Scripting engine: Lua 5.4
User data directory: C:\Users\chris\AppData\Roaming\pandoc
Copyright (C) 2006-2022 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

This could be probably using some flags or environment variable to opt in adding this string output and that would be set in the Github action build workflow

pandoc/src/Text/Pandoc/App/CommandLineOptions.hs

Lines 946 to 962 in a9498a1

	, Option "v" ["version"]
	(NoArg
	(\_ -> do
	prg <- getProgName
	defaultDatadir <- defaultUserDataDir
	luaVersion <- HsLua.run @HsLua.Exception $ do
	openlibs
	getglobal "_VERSION"
	peek top
	UTF8.hPutStrLn stdout
	$ T.pack
	$ prg ++ " " ++ T.unpack pandocVersion ++
	compileInfo ++ "\nScripting engine: " ++ luaVersion ++
	"\nUser data directory: " ++ defaultDatadir ++
	('\n':copyrightMessage)
	exitSuccess ))
	"" -- "Print version"

Regarding alternatives I have considered, our current workaround is to set an environment variable indicating we are installing a nightly build version, and check this instead of the version number only in our conditional testing. This is not ideal it requires manual editing when a new version is released to adjust as pandoc --version will output a higher version in the new release.

I am opening this issue for discussion. Is this something you would consider adding ?

Thank you.

[jgm]

I don't want to mess with version numbers (which aren't really meaningful for the nightlies anyway, since we often don't increment them with API changes until a release happens).
But including the git hash in the --version output seems a good idea. Indeed, we could do this for all builds, not just nightlies: why not?

[jgm]

Note: some builds aren't from a git repository, but e.g. from a tarball, so maybe not all builds.

[jgm]

I found https://hackage.haskell.org/package/githash

[cderv]

If we have a way to differentiate easily an unreleased build version from a release build, then it would improve this.

If it is not by the version number, that is fine. It means the latest release, and unreleased build would still have the same version though. If the later have a hash included, that would be enough to parse the version message and know this is not a released build but a newer version.

And if some builds have that, even if not all, that would be good too.

Thanks for considering

[jgm]

I've pushed a commit that adds info about the git hash and date of the build to all builds.
This doesn't address your desire to have a clear marking of whether it's a release build, so perhaps we can keep this open for now.

[jgm]

I've reverted that commit and used a different approach, just adding the suffix -nightly-DATE on the version for nightlies. (This is triggered via a cabal flag nightly.)

[cderv]

That will work perfectly thank you !

[cderv]

I don't know if this is directly related to this change but could be. the nightly version does not work correctly:

❯ echo "**test**" | ./pandoc -t native
Could not find data file D:\a\pandoc\pandoc\.stack-work\install\05458cf5\share\x86_64-windows-ghc-8.10.7\pandoc-2.18\data\abbreviations
❯ ./pandoc --version
pandoc.exe 2.18-nightly-2022-04-12
Compiled with pandoc-types 1.22.2, texmath 0.12.5, skylighting 0.12.3,
citeproc 0.7, ipynb 0.2, hslua 2.2.0
Scripting engine: Lua 5.4
User data directory: C:\Users\chris\AppData\Roaming\pandoc
Copyright (C) 2006-2022 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

Seems like some hard coded path from the Windows build are there now. Do you want a new issue for this ?

[jgm]

I can guess what the problem is...it seems to be overwriting all flags rather than just setting teh nightly flag. We can reopen this.

[jgm]

Let's see how tonight's nightlies look.

[jgm]

Oh, I just realized that the patch I suggested is completely wrong. (It will print the current time, not the compile time.)
We need either template haskell or something simpler, like passing in an explicit version suffix using a CPP define.

[jgm]

For now I've rolled back all the changes.
Will try another approach later.

[cderv]

Ok thanks. I'll revert on my side too then.

[hseg]

This doesn't address your desire to have a clear marking of whether it's a
release build

Judging by git describe's manpage, the problem is that githash neither
exposes git describe --exact-match, nor does it decompose the structure
returned by git describe. It could do so, by instead running
git describe --tags --abbrev=0,
git rev-list $(git rev-list --tags --no-walk --max-count=1)..@ --count,
git rev-parse HEAD respectively. We could then detect non-release builds by
checking if the commit count is 0.

Or alternatively, we could parse git describe's output ourselves, eg

commitsSinceTag :: GitInfo -> Maybe Int
commitsSinceTag gi =
    read . rtakeWhile (/='-') <$> lstrip ("-g" ++ giTag gi) (giDescribe gi)
  where
    lstrip xs ys = reverse <$> strip (reverse xs) (reverse ys)
    strip [] ys = Just ys
    strip (x:xs) (y:ys) = guard (x == y) *> strip xs ys
    rtakeWhile p = reverse . takeWhile p . reverse

Much less elegant. Do we want to bug githash upstream about this?

[jgm]

Actually the githash package provides giTag, which does list the number of commits since the last tag.

I didn't want to use this, though, because I usually tag AFTER building releases (to make sure they work). So this method can't detect a release build, unless we build the packages again after tagging, which i'd rather not do (waste of energy).

Anyway, I've pushed a new patch which builds in the compile date when the nightly flag is set, and without requiring the githash package.

[hseg]

Right, that's what I suggested doing in my snippet. Though am afk, so I can't check if I didn't have `giTag` and `giDescribe` mixed up in it. Have you had things break on the "bump cabal version" commit? Otherwise, you could test, then do that commit, tag, and rebuild. Alternatively (though problematically), you could tag the "update changelog/manpage" commits. In any case, this solution works well enough that you can ignore these comments if they're useless. Am 13. April 2022 04:08:09 GMT+03:00 schrieb John MacFarlane ***@***.***>:

…

Actually the githash package provides giTag, which does list the number of commits since the last tag. I didn't want to use this, though, because I usually tag AFTER building releases (to make sure they work). So this method can't detect a release build, unless we build the packages again after tagging, which i'd rather not do (waste of energy). Anyway, I've pushed a new patch which builds in the compile date when the `nightly` flag is set, and without requiring the githash package. -- Reply to this email directly or view it on GitHub: #8016 (comment) You are receiving this because you commented. Message ID: ***@***.***>