Please bump pandoc-cli version to something newer than 2.19.2

[jonassmedegaard]

Until release 2.19.2, same version was used to track library and command-line tool.
After that, library was bumped up to 3.0 whereas command-line tool was bumped down to 0.1.

Please bump up command-line tool to a version newer than 2.19.2, to ease upgrading on systems tracking version numbers for executables and not permitting version numbers to ever be lowered - e.g. Debian and its derivatives.
Alternatively, Debian will either need to rename package for executable from "pandoc" to e.g. "pandoc-cli", or introduce a so-called "epoch", meaning that all future releases in Debian will carry a prefix - e.g. "pandoc-2:0.1.1.1". Neither of those alternatives seem as attractive as a one-time upstream bump of version number for the "pandoc-cli" project.

[jgm]

We're following the Haskell PVP here. The version number of pandoc-cli reflects API changes in that package. Keeping it in sync with the version of pandoc would (a) depart from the PVP and (b) require frequent releases of pandoc-cli, with changes in version, with no code changes in pandoc-cli itself. That's quite bad from a Haskell point of view.

Perhaps a solution for Debian would be to add the pandoc-cli subdirectory as a patch to the pandoc source, instead of treating it as a separate package?

[jgm]

I suppose another possibility would be to deprecate pandoc-cli and go back to making pandoc a package that includes both a library and an executable. I think the main motivation for splitting the old pandoc into pandoc, pandoc-server, pandoc-lua-engine, and pandoc-cli was to allow pandoc to be built without lua and/or server support. pandoc-cli itself doesn't really introduce new dependencies.

Probably we should have done it this way to begin with, but I'd have a lot of worries about causing inconvenience for everyone by changing this again.

[jonassmedegaard]

What I propose here is not to keep pandoc-cli in sync with pandoc, but instead to once bump the version number of pandoc-cli.

The problem is not library and program being out of sync. It is having a program that used to be released as 2.x now being released as 0.x.

Concretely I propose to do a one-time change of pandoc-cli bumping it to 3 (or 3.0.0.0, 3.1.1.2 or 2.19.3 - whichever you like the most, as long as it is a number above the already existing `2.19.2 tied to a historical release of the executable binary).

[jgm]

One of us (or both) isn't understanding the other. Let me be more explicit. When we do a new pandoc release, it's quite frequently the case that pandoc-cli does not have any changes and hence does not have a bump in version. Only pandoc (which contains the library and basically all the functionality except server and lua parts) gets a version bump. The version reported by the pandoc executable is the version of the pandoc package.

I agree completely, pandoc should not be released as 0.x. It should be released with the version specified in pandoc.cabal. What I don't understand is why a 0.x version on the pandoc-cli package is preventing this.

If the reason for this is that Debian needs the version of pandoc-cli (which supplies the pandoc executable) to be the same as the version of the pandoc release and the version reported by pandoc --version, then yes, frequent bumps in pandoc-cli's version would be required; a one-time bump would not be enough.

[jgm]

The more I think about it, the more I want to get rid of pandoc-cli and just restore the executable stanza in pandoc. It is definitely awkward to have a source package with one version number that installs an executable that reports a different version number. And many people were confused when cabal install pandoc no longer installed the executable.

EDIT: Never mind, that isn't possible, because then we'd have cyclic dependencies: pandoc-server depends on pandoc, and pandoc-cli depends on pandoc-server.

[alerque]

For reference we had the same issue as Debian over on Arch Linux. Our resolution was to introduce a pandoc-cli package dependent on haskell-pandoc, but setup the upgrade path so that the CLI package replaces the now obsolete plain pandoc package while also providing pandoc so than existing dependencies don't break.

[jonassmedegaard]

I think the confusion stems from our different naming of packages and project(s). Let me try a different approach:

Older library, upstream: pandoc 2.x
Older library, Debian: libghc-pandoc-* 2.x
Older binary, upstream: pandoc 2.x (sync with library)
Older binary, Debian: pandoc 2.x (sync with library)

Current library, upstream: pandoc 3.x
Current library, Debian: libghc-pandoc-* 3.x
Current binary, upstream: pandoc-cli 0.x
Current binary, Debian: pandoc 0.x (same package name as before, same version as upstream forbidden in Debian for versions to go "backwards"
Current binary, Debian: pandoc 2:0.x (same package name as before, version with epoch annoying to need to remember for eternity that Debian versions are in a different 2: prefixed namespace
Current binary, Debian: pandoc-cli 0.x (new package name, same version as upstream non-intuitive for Debian users that binary moves to a different package

Debian users are best served if they can continue to install package "pandoc" to get the binary - it is perfectly fine that upstream has split development into multiple prockects/sections/chungs/whateverthename, and perfectly fine that library and binary versions are not kept in sync, but it is problematic that the newly introduced thingy begins at a version lower than what the older equivalent thingy was versioned.

What I request is a one-time bump of project pandoc-cli so that Debian republishing as pandoc can be done without adding an epoch, e.g. like this:

Next library, upstream: pandoc 3.x (can be unchanged, i.e. no need to bump library at all!)
Next library, Debian: libghc-pandoc-* 3.x
Next binary, upstream: pandoc-cli 3.0
Next binary, Debian: pandoc 3.0 (same package name as before, same version as upstream

Hope this makes better sense

[jgm]

I think people will be extremely confused if the Debian pandoc-3.0 package installs pandoc 3.1.2 or whatever...

If needed, I can move to keeping the version of pandoc and pandoc-cli in sync (violating the PVP for pandoc-cli). Pandoc probably wouldn't be what it is today if Debian hadn't picked it up very early in its existence. This would require a tiny bit more work from me on releases.

[jonassmedegaard]

I think users will be only mildly confused, and going forward you can make use of the separate version numbers to track changes to features for the library and changes to the command-line options for the binary.

More precisely, what I will do (if you choose to bump pandoc-cli to 3.0) is to release that for Debian as pandoc-3.0-1+3.0.1 - which means binary version 3.0, Debian revision 1, linked against library version 3.0.1.
So those caring about version numbers will get more details, not less.

[alerque]

I kind of object to the half-way measure being suggested. I can see syncing the CLI package version to the library, or I can see just leaving it the way it is, but bumping it one time to an arbitrary version that seems to have meaning and then leaving it get out of sync again is going to be ever more confusing to end users, at least on the Arch side. At least now with the CLI having a number totally out of range for Pandoc as people know it they are not expecting it to be any particular exact version. If it were to be bumped arbitrarily to 3.1.2 and then left to get out of sync again that would be even more mess than if it just stayed in sync and always dependend on cli-version==library-version.

[jonassmedegaard]

The bump is not arbitrary, it is ensuring that the binary is counting upwards from before the split.

I don't mind bumping up to 659.22 instead - which would address Arch users being less confused by a wildly off version while at the same time addressing the Debian need of the version being bigger than the last version before the split. And that I would call an arbitrary number.

[jgm]

But it would certainly be nicer to have the debian package name pandoc-3.1.9 instead of pandoc-3.0-1+3.1.9, wouldn't it? I just worry that the X+Y thing will confuse people. "I installed pandoc 3.0, but I got something else!" And if it's pandoc-99+3.1.9 that might really be confusing.

[jonassmedegaard]

Debian users are already mildly confused when Apache 2.x.y does not link with a specific version of OpenSSL but with a varying one, to allow for security fixes to OpenSSL without at the same time doing feature-full updates to all of its reverse dependencies.

[jgm]

Well in the case of apache, at least the first component (before the +) is the actual version of apache being installed, and the part after is the version of a dependent library. You might say that, technically, the same would be true of pandoc. But there's a difference; I don't think of the pandoc-cli version as the version of the pandoc release or the version of the command line program; rather, it's the pandoc version. pandoc-cli is really the dependent library here, conceptually at least.

[jonassmedegaard]

You have a valid point, of tracking features as primary version component, and command-line interface only as secondary component.

I don't agree with your point, but it is valid regardless. Reason that I don't agree is that I would consider it a larger breakage if a command-line tool removed/renamed a command-line option, than if it removed a parsing or rendering feature.

It is technically possible for me to provide a Debian package "pandoc-3.0.1+0.1.1.1-1", i.e. flip around the logic for our users to have library version be primary component and cli version be secondary.

I would still ask to please have the cli version bumped, because the kinds of tricks I can make towards users (which is called a "binary package" in Debian lingo) is different from what I can do at compile time (called "source package"): I still need the source package to be a larger version than previously published.

[jonassmedegaard]

Oh, and just for completeness sake: Debian do not really need the pandoc-cli version to be higher than 2.19.2, only higher than 2.17.1.1 (the latest version packaged for Debian). I proposed newer than 2.19.2 to also cover similar needs for other redistributors.

[jgm]

Reason that I don't agree is that I would consider it a larger breakage if a command-line tool removed/renamed a command-line option, than if it removed a parsing or rendering feature.

Note that all of the command-line option parsing logic is exposed by the library and thus lives in the pandoc package. pandoc-cli doesn't have much actual logic in it; it just exists to integrate pandoc, pandoc-lua, and pandoc-server and create an executable.

[jonassmedegaard]

Ok. Is that a reason to keep versioning at 0.1.1.1...?

[alerque]

Yes, that follows PVP which this and most Haskell projects follow. It seems to me a strong argument would be needed to ignore PVP and just assign an unrelated number to facilitate some other issue. Personally I could see a rational for re-integrating the CLI package into the pandoc package, or maybe for tracking the version, but I still can't just a one time version sync with no semantic meaning that then later causes even more version confusion.

[jonassmedegaard]

but I still can't just a one time version sync with no semantic meaning

How do you reach the conclusion that there is "no semantic meaning"?

Semantically, version 0.1 means "new code project" or "this project release contains barely no differences from anything prior".

This is however a situation where a part of an existing project was branched off into a separate project.
My request is not that some arbitrary version gets surprisingly thrown into a weird mix for no semantic reason, on the contrary I request that the larger semantics of the collective set of projects is taken into account in the versioning of this new branch.

that then later causes even more version confusion

What "even more version confusion" are you talking about?

Since this is a branch, the needed version adjustment to align with the collective is a one-time action, not something that requires busywork for each future release of that collection of projects.

I envision no further need for future version bumps, until next time some part is branched off - which I expect to happen extremely rarely, if ever again.

[jonassmedegaard]

I really dislike having to rename the Debian source package. For all the concerns about changes to versions being confusing for users, I strongly believe that renaming the core name can only be more confusing.

And effectively changing the executable from a 2.x version to a 0.x version (and not "correcting" that again, as I request here) will force such renaming of package in Debian, because numbers bust be accumulative.

[jgm]

Ok. Is that a reason to keep versioning at 0.1.1.1...?

No. I'm ok with reversioning pandoc-cli if it's necessary for you. What I'm suggesting is that it might be better to sync version numbers with pandoc and pandoc-cli (and depart from PVP for pandoc-cli) than to have a strange Debian package name like pandoc-3+3.1.5. Not completely sure though. If you can make the main pandoc version the first component, as in pandoc-3.1.5+3, then that seems better, though it may still confuse some people.

But here's a question. Why do you need the +3 at all? Why can't it just be pandoc-3.1.5 as in the old days? Who cares what version the pandoc-cli package has? It's just a small shim to produce the executable; all of the functionality is in pandoc, pandoc-lua-engine, and pandoc-server.

[jonassmedegaard]

Why do you need the +3 at all? Why can't it just be pandoc-3.1.5 as in the old days?

When pandoc-cli is released as an independent project with an independent release cycle, then it needs to be treated as such: Its own version number needs to be included, to be able to distinguish e.g "pandoc-cli 0.1 linked with the library pandoc 3.1.5" from "pandoc-cli 0.1.1 linked with same library pandoc 3.1.5".

I can extend the version number used for the binary Debian package to include the version of a prominent dependent library (the pandoc library), but if I instead replace upstream version with upstream version of said prominent library, then it can lead to situations where a newly issued Debian package clashes with an already existing Debian package - that cannot happen: Each Debian package name+version must be unique.

[jonassmedegaard]

In principle, I could avoid introducing pandoc-cli as an independent package in Debian: I could embed pandoc-cli into the same Debian source package "pandoc", but then - again to ensure combinatorical uniqueness - that combined source package now used for both library and cli binary packages would need to include the version number of the embedded pandoc-cli upstream source.

Also, our helper tools in Debian to handle building of Haskell projects are geared towards one Debian source package for one upstream Haskell project, so it would be a nightmare to embed another project and tweak routines to handle two projects within one source package. But even with that solved, there would still be multiple moving parts requiring multiple identifiers.

[jgm]

If we were to synchronize versions for pandoc-cli and pandoc, so that each version of pandoc-cli depended on an exact version of pandoc with the same version number, would that avoid the need for the double numbering scheme or would it still be needed? (That's what I currently do with skylighting & skylighting-core, btw.)

[jgm]

Jonas, where do we stand on this? To summarize my positions:

• As noted, I'm willing to bump the version of pandoc-cli.
• I'd like to make a decision now as to whether to bump to a relatively stable version (e.g. 3) that could stay unchanged while the pandoc version changes, or to adopt a policy of synchronizing pandoc-cli and pandoc versions. It would be good to have your feedback on whether that would matter at all for Debian packaging.
• If it's possible to have the scheme pandoc-{pandoc version}-{revision}+{pandoc-cli version}, that seems less likely to confuse people than the other way round.

[jonassmedegaard]

You could promise a policy of always releasing pandoc-cli and pandoc totally in lockstep - i.e. if a release with only a comma changed to one part, then both parts would be be released with that new version number - even though the other part would have zero changes. Then you would not even need to ask whether we Debian would accept that or not: We would simply package pandoc-cli with its version number, which would happen to be identical to the pandoc version number - no need for Debian to do anything special.
But you would then do something odd upstream: You would issue a pile of releases of pandoc-cli with zero changes. Feel free to do that, it just feels quite suboptimal to me, and the reason I proposed to not keep the version numbers in sync.

If it's possible to have the scheme pandoc-{pandoc version}-{revision}+{pandoc-cli version}, that seems less likely to confuse people than the other way round.

Sorry, but no can do: A Debian version string consist of package name + upstream version + Debian revision, where upstream version is bumped every time the upstream source changes, and the Debian revision changes is bumped every time the Debian packaging changes. I can "distort" upstream version and/or Debian revision, e.g. by adding a prefix or a suffix, but each need to be bumped (and only ever upwards!) as is their function.

Prepending pandoc version to upstream version is ok, and appending pandoc-cli to Debian revision is ok as well, but suppressing pandoc-cli version from upstream version means that there is no guarantee that the number gets bumped at each upstream source change.

I cannot release pandoc-cli as Debian package pandoc-0.1.1.1-1 because that is lower than previously released pandoc-2.17.1.1-3.

Simplest is that I release package pandoc-2.17.1.1+0.1.1.1-1 but I would find that confusing.

I can instead release package pandoc-3+0.1.1.1-1 - i.e. locally in Debian add prefix "3+" to upstream version, to indicate compactly that it is the pandoc executable in the era of 3.x. I don't need you to do anything for that versioning scheme.

What I find more elegant is to release package pandoc-3-1 but for that I would need you to bump pandoc-cli from current 0.1.1.1 to 3. Once only, after that you continue to track pandoc and pandoc-cli independently as you are doing now, which makes good sense to me.

Sure, you can instead choose to bump pandoc-cli to 3.1.5 and I would release that as package pandoc-3.1.5-1. And if you then bumped pandoc and pandoc-cli in total lockstep then effectively it would look like the Debian package upstream version were reflecting the library version - but if Debian at some point chose to deviate and patch one version of pandoc-cli to link against a differently versioned pandoc then the version would reflect the cli version, not the library version.

Yeah, I could then sprinkle library version into the package version - but because that would be tied not to the source of pandoc-cli but to the packaging of it (build-time linking and potentially patching), the library version should be added as prefix or suffix of the Debian revision. I think it might be helpful for some but confusing for most.

[alerque]

Simplest is that I release package pandoc-2.17.1.1+0.1.1.1-1 but I would find that confusing.

This seems like by far the least confusing and also semantically correction option being proposed to me. The debian package revision being at the end is only a minor variation on what John proposed and makes sense in the distro packaging scheme world: {package-name}-{package-version}+{supporting-package-version}-{distro-release-iterator}.

[jgm]

I agree: pandoc-2.17.1.1+0.1.1.1-1 is by far the least confusing option of those suggested. And if I understand correctly, it doesn't even require a bump in pandoc-cli's version number?

[jonassmedegaard]

so you don't find it confusing that upstream version part of Debian pandoc package is stock forever at 2.17.1.1?

[jonassmedegaard]

You really find it "the least confusing" that pandoc 5 years from now, with upstream library version 4.23.2.8 and cli version of 1.6.0.1 is shipped in Debian as pandoc-2.17.1.1+1.6.0.1?

[jgm]

OK, I must have misunderstood. I thought 2.17.1.1 was just standing in for the pandoc version, and you meant that the versioning scheme would be:

pandoc-{hackage pandoc version}+{hackage pandoc-cli version}-{revision}

That's what I'd hoped for, and in an earlier comment in this thread you seemed to say it was possible:

It is technically possible for me to provide a Debian package "pandoc-3.0.1+0.1.1.1-1", i.e. flip around the logic for our users to have library version be primary component and cli version be secondary.

Anyway, that's exactly what I want. You said above you'd still want a one-time bump in the pandoc-cli version. If it's really necessary, I could do that. The important thing is that when someone wants to install pandoc version 3.5.99, they will do it by installing a debian package pandoc-3.5.99+something.

[jonassmedegaard]

Indeed, I wrote that it was possible to include library version as prefix for upstream Debian version.
I later realized that it is not robust to do that - where my concern is not only upstream versioning, but also the ability for the Debian packaging to be flexible (e.g. cherry-pick patches that might lead to different lig+cli combinations than released upstream).

...but it seems there is no solution both user-intuitive and also robust, and I should let go of striving for that.
I will now try a variant of the "non-robust" approach not requiring a one-time bump:
Debian source package: pandoc-2.17.1.1+{hackage pandoc-cli version}-{revision}
Debian binary package: pandoc-{hackage pandoc version}+{hackage pandoc-cli version}-{revision}

This means concretely that an end-user would do apt install pandoc-3.0.1+0.1.1.1 to get the Debian-packaged pandoc-cli 0.1.1.1 linked against pandoc 3.0.1, but a developer wanting to inspect or recompile that same package would do apt source pandoc-2.17.1.1+0.1.1.1.

This approach is robust for source package naming and theoretically non-robust for binary packages, which is acceptable for me.

Thanks for taking the time to discuss this. Sorry that in the end it seems the discussion was all for nothing - it wasn't for me, and I still find it peculiar that pandoc-cli has a 0.x version but I do understand now the logic for that.

So to reiterate: I do not need any one-time bumping of pandoc-cli version after all.

edit: Current approach involves prefixing Debian source package with the static string "2.17.1.1+".

[jonassmedegaard]

Correction: my current approach involves prefixing Debian source package with the static string "2.17.1.1+".

[jgm]

I like the approach to binary package naming, but it does seem potentially very confusing to have different versioning for the source package, and especially to have versioning for the source package that looks like a pandoc version number but isn't...

As a minimal change to your idea, could you make the source package pandoc-3+{hackage pandoc-cli version}-{revision}? This would still increment the number, but the 3 is less likely to be confused with a pandoc version number.

I will keep this open, because I would still like to think about the option of synchronizing pandoc and pandoc-cli versions. You say:

But you would then do something odd upstream: You would issue a pile of releases of pandoc-cli with zero changes.

Not zero. Each version of pandoc-cli would depend on an exact pandoc version. So, when we release pandoc X.Y.Z, we'd change pandoc-cli.cabal to depend on pandoc == X.Y.Z. The version scheme for pandoc-cli would not follow the PVP, but I think it could be motivated, given the aims of pandoc-cli.

The advantage of this approach is that a user could install pandoc X.Y.Z by doing cabal install pandoc-cli-X.Y.Z. That isn't possible now. If you do cabal install pandoc-cli, you'll get some version of pandoc in the range allowed by pandoc-cli, but it isn't guaranteed to be the latest version. You can target a specific version by doing cabal install pandoc-cli --constrain pandoc == X.Y.Z but that's less straightforward.

I'd be interested in feedback from others on this idea.

[jonassmedegaard]

As a minimal change to your idea, could you make the source package pandoc-3+{hackage pandoc-cli version}-{revision}? This would still increment the number, but the 3 is less likely to be confused with a pandoc version number.

I want to get rid of the dirty hack of prepending some static number to "lift" pandoc-cli to have version higher than what it used to have when released together with the library.

Using the minimal possible static prefix "2.17.1.1+" I can hope that pandoc-cli some day in perhaps 5-8 years to naturally evolve beyond that number, and I can drop the prefix.

If I instead used the prefix "3+" then I would have to maintain that dirty hack for perhaps 10-13 years.

This is the reason I opened this issue: If you - today - move pandoc-cli to a version higher than 2.17.1.1, then I need no dirty hack at all.

[jonassmedegaard]

Arrgh - apparently man pages are maintained not as part of pandoc-cli but as part of the library.

I have now changed approach for Debian packaging of the executable binary to trach Github source commits, distributing whatever commit seems more likely to contain pandoc-cli code and man pages suitable for the version of the (separately packaged in Debian, due to limitations of Debian helper scripts affecting the ability to build multiple Haskell projects as a single Debian source package) pandoc library, using naming scheme pandoc-2.17.1.1+{Github commit date}-{revision} for the Debian binary package.

[jgm]

Arrgh - apparently man pages are maintained not as part of pandoc-cli but as part of the library.

That makes sense, because all of the option parsing code is in the library. Remember, the library implements everything. The pandoc-cli package is just like "glue." It's an auxiliary package for building the executable.

Tracking github source commits for the source package is a very sensible approach.

whatever commit seems more likely to contain pandoc-cli code and man pages suitable for the version

We use git tags for releases, so for the X.Y.Z release of pandoc you can use the tag X.Y.Z.

[jgm]

I'm going to move to synchronizing the version of pandoc-cli with pandoc. Each pandoc-cli version will depend on a precise version of pandoc. This is a harmless violation of the PVP because pandoc-cli doesn't export a library. Man pages will live in the pandoc-cli source distribution.