What do the versions mean?

[dAnjou]

Hi,

when I read versions in the format of x.y.z I assume semantic versioning (maybe I shouldn't?). But it seems gopass introduces quite a few breaking changes in new minor version.

So, what do gopass' versions mean?

Background is that we rely heavily on gopass in a team of ~40 people for our infrastructure, system, and application secrets. We maintain our own homebrew repo which provides a specific version of various tools, gopass included. The other day I updated that version from 1.8.6 to 1.9.2 assuming everything would just continue working. But we ran at least into issue #1353 which is kind of a bummer because now we'll have to update a lot of repos and communicate that to a lot of people, or we go back and are stuck with 1.8 for now.

[dominikschulz]

This is a good question. Thanks for asking.

First of all gopass is using semantic versioning because that's what a lot of tools, e.g. goreleaser expect.
We try to adhere to the spirit of semantic versioning, i.e. no breaking changes in patch releases, no major architectural changes in minor releases.

But what's a breaking change? Any change might break someones workflow.
So I'd say a breaking change means changing the previously defined behaviour.

However we never did a clean definition so far. This is something we started to address while working on the 1.10.0 release.
Now you'll find documentation / specification for all supported CLI commands in docs/commands.

We've also started to define our core use cases - however that work isn't finished, yet.

So I'd suggest you look at the existing command documentation and file issues if you notice any discrepancies between what's documented and actual behaviour on the one hand as well as discrepancies between your expectations and the specification on the other hand.

Then we can start to discuss specific details.

One thing we should do to resolve this issue is to document how we define our versioning scheme.

[dominikschulz]

Oh, and btw. what's the problem with #1353 exactly?
Maybe you want to re-open that issue if it's still affecting you or file a new one detailing your conerns.
But keep in mind that a lot ton of issues were fixed in 1.10.0. Did you try that one?

[dAnjou]

Thanks for responding so quickly! There are a few things to unpack, I hope I manage to address them all.

We try to adhere to the spirit of semantic versioning, i.e. no breaking changes in patch releases, no major architectural changes in minor releases.

When you say "semantic versioning", are you referring to semver.org or a versioning schema that adheres to your own semantics? Either is fine, I'd just like to be on the same page.

The semantics of minor releases you're describing don't make much sense to me. In my understanding of software, architectural changes don't necessarily change or break the API or UI. So, when would those be okay?

And isn't a new secrets format a major architectural change? Especially because you're suggesting in the changelog:

you might want to run gopass config mime false before performing any kind of write operation on the password store. Otherwise mutated secrets will be written using the new native gopass MIME format and might not be readable from other implementations.

That sounds pretty disruptive to me. Because - like in our case - when some people update to 1.10.0 and others are still on 1.8.6 things are bound to break for them. You could argue that people should always keep their tools up-to-date but with a lot of people this is hard to coordinate. For scenarios like this it'd be good to announce such changes via the tool itself, as in, you release a version that keeps the behavior but announces its deprecation. Much like in libraries and frameworks that log deprecation warnings but keep that behavior for a few versions.

However we never did a clean definition so far. This is something we started to address while working on the 1.10.0 release.
We've also started to define our core use cases - however that work isn't finished, yet.

That's a very good thing. It's a bit unfortunate that gopass' versioning started with 1.0.0. According to semver.org it would have been better to start with 0: Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable. But it is what it is now.

So, all that being said, why not go to 2.0.0 instead of 1.10.0?

Oh, and btw. what's the problem with #1353 exactly?
Maybe you want to re-open that issue if it's still affecting you or file a new one detailing your conerns.
But keep in mind that a lot ton of issues were fixed in 1.10.0. Did you try that one?

It's just something that accidentally broke people's setup, ours included. We'll find a way around it but not by updating to 1.10.0 because I'm too afraid it'll break too many things for too many people and too many repos containing scripts using gopass.

[dominikschulz]

I'm referring to semver.org. But we never wanted gopass to be used as a library and didn't hold ourselves accountable to the strict versioning style.

Maybe we should still be on a major version of 0. I'm not sure.
Btw. we started interally with a major version of 0.

The upgrade path for the new secrects format is indeed a bit rough. That wasn't planned very well and I wonder if it's to late to revert the default to not using the new format, yet. We still support all previous formats, but you're right. Users of older versions will get into trouble. That wasn't intentional.

Maybe 1.10.0 should have been a 2.0.0 relase. But I guess that's too late now.

I'd appreciate if you could help us (e.g. by providing feedback) making the next 1.10.x releases one you'd be willing to upgrade to. I'm sure others would appreciate that, too.

[AnomalRoil]

For #1353 the problem actually stem from the fact that autoclip should not have influenced show and it should now have been fixed in #1513

1.10 should actually be easier to migrate to than 1.9.2, I think since we corrected a lot of bugs / misbehaviour that were in 1.9.2 since it wouldn't suffer from #1353 for instance.

If you have a big team using gopass, we would be really super happy to get as much feedback as you can give us :)

[dAnjou]

Thank you for the insight, @dominikschulz. And thanks for the info and encouragement, @AnomalRoil. Sorry for the late response. I've moved on to another project, so I can't really support my old project with upgrading gopass anymore, I hope when they try that they'll find this thread.

Regarding feedback, I do have a few things (this is for version 1.8.6). I don't know if this thread is the right place, if not I'm happy to repost somewhere else:

• Over time, years in our case, the store becomes almost unmanageable big, both in number of secrets (I'll get to some reasons later) and in storage size, gigabytes in our case. Adding new recipients takes half an hour with reencrypting all the secrets (only recently we discovered the concurrency setting). It became a gamble because - I don't know why exactly - we had cases where someone was in the process of adding a new recipient which then failed because another person had updated the store in some way in the meantime. It had something to do with the Git commits, I think. Eventually we decided to cut off a big chunk of the Git history. But that means you lose some of the audit trail, not tragic in our case but also not ideal, I think.
• I mentioned the sheer number of secrets. We ended up with quite a few duplicated entries because some secrets, like certificates, were used in multiple places. This is definitely partly a problem on the user side, we should have been more disciplined or strict or should've come up with a policy of how to organize secrets. But maybe there's a way the tool could assist.
• Discoverability was another issue, and also a reason why people created duplicates. But simply finding credentials for something was difficult sometimes because the creators chose a name that was intuitive for them but not for others. So, maybe metadata like tags could help here when they are included in the search. The new format might enable this.
• Another super helpful feature would be secret deprecation or expiry. There are multiple reasons for this. The obvious one is that credentials should be changed now and then, especially considering that people, like me, leave the team/project or the company altogether (such an event should trigger a complete credentials rotation anyway but that's a whole different story). Another reason is that sometimes credentials simply become obsolete, we've accumulated a lot of zombie credentials in our store over time. And there was not really a way for us to know which ones are still used. At some point I even thought about implementing a wrapper that logs accesses, and let people install and use that instead of gopass directly. Such a feature would also help with the other things I mentioned before.
• Something that came up recently was that we couldn't add a secret because one recipient's key had expired. For some reason I didn't have to deal with that in all my time in the project before. But when I had to I became quite confused, and it's probably partly because I don't fully understand the concepts behind gopass, or gpg for that matter. Fortunately we could quickly contact that person, not a tech-savvy one btw., and helped them update their key. But what if that person had been on holiday? After updating the key and syncing it via gpg we tried to update the recipient, or rather add them again (what does update do?). But it said that the recipient existed already. What's the workflow in this case? Eventually we simply removed and re-added them which meant that we had to reencrypt all secrets again. I guess that's how it's supposed to work, is it? It was quite confusing though.

Maybe some of these things have been mentioned by other people in other issues before, I didn't check. It's just the major pain points I can remember from working with it in a bigger team for almost 2.5 years.

All of this being said, I know that gopass solves a difficult problem and I'm aware that good security often comes at the cost of convenience. Some things I mentioned are just rough edges that can be polished. But maybe there's also a deeper issue with Git as the default backend, I don't know, just guessing.

[dominikschulz]

Thank you very much for sharing your experience with us. Actually with 1.11.0 some breaking changes have been rolled back and other issues have been fixed, so upgrading from older versions should have become feasible again.

gopass was never designed for huge teams or huge repo sizes. If most people are only reading from the repo having three digit number of recipients should be OK, but anything more than that is not something we can or want to support. For that you'd need a different system. I have worked with decent web-based credential managers, but of course they come with their own set of risks and no decent open-source ones seem to exist (and commercial ones are not an option for me in this use case).

Expring recipient keys is one of the issues that come with using gpg. I've tried to get rid of that, but too many people need use gpg so we're stuck with that. We'll need to figure out the best option to deal with that.

[dominikschulz]

@dAnjou Please note that many of these issues should have been addressed meanwhile and we tried to properly document our promises around API stability and what versions mean. Also we have taken a much more cautionous approach wrt. breaking changes will be more careful going forward. The catch is that you'll have to upgrade to 1.11.0 (or better: 1.12.0+) to get there.

Feel free to re-open in case this is still and issue that needs to be discussed.