Versioning

[rye]

We should start thinking about versioning this using SemVer. The two questions I can think of are:

• Do we want to version this project? Or should this be rolling-release?
  • If so, what will be our indication that we are ready to release version 1?

This might affect our update-checking strategy for #56; users could also use a variable to decide whether they only want to use the latest release or the Git HEAD. (master in our case)

If we decide that we already passed a point where the project could be considered at v0.x.y, we can go back and make releases now for those points in history, and I can do that.

[rye]

This will also allow us to use milestones going forward, which is nice.

[cg505]

rolling-release bruh

[rye]

rolling-release bruh

While I like the idea of rolling-release, having some idea of stable vs development would be smart. SemVer doesn't impose much overhead, and it's fun to release new versions.

Versioning is important especially to users who don't want to update every time their editor starts, and who are less likely to report and tolerate issues as they happen. Milestones are a better way of structuring work, as well.

It's important to note that I have some people who are using this configuration here at St. Olaf, so it would make sense to keep master as a stable/development branch (tested on CI via #53) and create versions every so often.

There's certainly nothing stopping us from tagging previous commits and establishing an existing version structure, either.

[cg505]

Okay, we can semver but also have a rolling-release system that works well?

[rye]

What exactly do you mean by a rolling-release system? I think it's totally fine to release new versions of dot as new stuff gets proven stable in master, so in that way we'd be rolling-release. But we'd just do semver rather than having an incrementing number. And master would always be the latest development version.

[cg505]

So... here's my idea.

As far as semver: a semver breaking change in our context is something that might cause you to have to change something in your personal config. (This is not completely without exception, but should generally be followed.)

• master branch: Has all the latest changes, and should always be in a stable state. Will eventually become the next minor-x.x branch.
• stable branch: Has the most recent semver release. Should always mirror the latest minor-x.x branch.
• minor-x.x branches for each minor version that allow us to release hotfixes to old minor versions that people might still be using and don't want to have to update their config yet. (Branch off the oldest minor-x.x branch that we want to give the hotfix for, make the hotfix, merge the hotfix branch back into all minor-x.x branches that need the hotfix.)
• feature/xyz branch: developing a new feature/whatever. Will be merged into master. Can have subbranches if you want/feel is necessary.
• hotfix/xyz branch: a hotfix that will potentially be released into multiple minor-x.x branches, and will definitely be merged into the most recent minor-x.x (aka stable) branch as a new release.

The user can choose to be on the stable branch or dev branch. If you're on dev, you get offered updates whenever anything ends up on master. Otherwise, you get offered updates whenever there is a release more recent than the one you're currently using. If you're on an old minor branch that has both a hotfix version bump and a minor version bump available, you should be able to choose which you want.

This is very rough and I came up with it in like 15 minutes. It's essentially the most straightforward application of semver/git I can think of to this project. Please leave recommendations.

[rye]

As far as semver: a semver breaking change in our context is something that might cause you to have to change something in your personal config. (This is not completely without exception, but should generally be followed.)

Agreed. Any change introduced by development that changes the "API"—the set of functions and behavior available to users.

• master branch: Has all the latest changes, and should always be in a stable state. Will eventually become the next minor-x.x branch.

This makes a lot of sense—"should always be in a stable state" is true. master will be the latest "development" version, but "development" for us should mean well-tested and CI-green changes only. (I'd suggest locking master to help enforce this.)

• stable branch: Has the most recent semver release. Should always mirror the latest minor-x.x branch.

I'm hesitant to keep a persistent branch, but I'm open to this. Perhaps latest would be a better name, but this would work. The alternative is to use the GitHub API. (/releases/latest maps to the latest release, but I'm not sure if that's what we want.)

• minor-x.x branches for each minor version that allow us to release hotfixes to old minor versions that people might still be using and don't want to have to update their config yet. (Branch off the oldest minor-x.x branch that we want to give the hotfix for, make the hotfix, merge the hotfix branch back into all minor-x.x branches that need the hotfix.)

As far as this goes, that makes sense. I'm not sure about leaving these branches lying around. My philosophy as far as version branches seems to be in favor of creating a new branch to release a new version, merging hotfixes into that branch, and doing that. Maybe vX.Y rather than minor-X.Y?

So when it comes time to release a new minor release, we'd create a branch to bump the version and we'd put a release/tag on that branch. (We can make drafts and collaborate to write descriptive release notes, and we should mention Issues/PR's and stuff, too.)

If there is an issue with a specific version in the past that is not a problem in master, we can create a hotfix for that specific version and release a new patch version by merging back onto that minor version's branch.

• feature/xyz branch: developing a new feature/whatever. Will be merged into master. Can have subbranches if you want/feel is necessary.

Perfect! I'm hesitant to use prefixes like feature/ but I definitely see the merit and I'd be glad to adopt it for this project.

• hotfix/xyz branch: a hotfix that will potentially be released into multiple minor-x.x branches, and will definitely be merged into the most recent minor-x.x (aka stable) branch as a new release.

Ditto here.

The user can choose to be on the stable branch or dev branch. If you're on dev, you get offered updates whenever anything ends up on master. Otherwise, you get offered updates whenever there is a release more recent than the one you're currently using. If you're on an old minor branch that has both a hotfix version bump and a minor version bump available, you should be able to choose which you want.

This makes a lot of sense. I'd love something that doesn't require us to keep persistent branches going, but if that creates the easiest workflow, I'm all for it. I'm also aligned with the idea of using tags to maintain points in history, then using those tags as bases for branches if we need to add a patch version.

[cg505]

Glad we are mostly in agreement!

I'm hesitant to keep a persistent branch, but I'm open to this. Perhaps latest would be a better name, but this would work.

This is basically just for convenience. latest would be fine too.

Maybe vX.Y rather than minor-X.Y?

The reason I didn't do this is that I was worried about ref conflicts between branches and tags. If you don't think that'll be an issue then I would also prefer the vX.Y syntax.

The real reason this thing is so longwinded is because I was thinking we would still be fixing bugs on old minor versions. If we don't need to be able to do this then we definitely don't need the branches lying around. The releases would do fine.

I'm hesitant to use prefixes like feature/ but I definitely see the merit and I'd be glad to adopt it for this project.

I'm fine with dropping the feature/ prefix. Actually, I like that, it was just sort of instinct.

[rye]

This is basically just for convenience. latest would be fine too.

Let's use latest then.

[...] I was worried about ref conflicts between branches and tags. If you don't think that'll be an issue then I would also prefer the vX.Y syntax.

That makes sense. Magit doesn't like it when there's a branch and a tag with the same name, but the actual Git CLI will properly handle it. For convenience and lack of confusion's sake, I'd recommend using release- as a prefix for branches to house any release work. (I.e. accepting PR's, adding changelog entries if we decide we want a changelog, bumping a version, that kind of thing.) This is a scheme I've seen in the wild; the release- branch can get discarded after the release is released and the branch is in master.

[...] I was thinking we would still be fixing bugs on old minor versions.

This [fixing bugs] is something that I think we should be prepared to do, and that I think we could do retroactively by creating branches off of our tags that already exist. Effectively a hotfix/ branch could be branched off of a tag, and then when a PR is submitted we could create a release-vX.Y.Z branch to also bump the version in addition to accepting the PR. A collection of PR's could go together into this release-vX.Y.Z branch.

It's worth noting that if we are getting into the thick of things and we have a fix that is incompatible with master, we'll have to get crafty by creating a separate development branch for that version.
It's probably best to avoid doing this—this is really something that only happens in bigger OSS projects where you have Debian-y users who adamantly refuse to update.

I'd much like our preference to fall towards solving issues off of the latest minor release only, to avoid this problem entirely by keeping everything synchronized. Then, we don't have to decide when a release is "too old," we could just assume that all people should update to the latest version.

So, with that all in mind, how about fire up a^{EDIT: Already exists; we will add as a new section.} CONTRIBUTING.md file to detail some of this scheme, and also to detail our process?

[cg505]

sounds good

[rye]

The work for adding versioning/branching guidelines will get thrown into the branch PR for #58, I think—that'll be an added section.

[rye]

Preliminary work to add this to the CONTRIBUTING.md file has been done over at #60.

[cg505]

We probably need to do a better job of specifying how we work with SemVer. For instance, what is an API change in our context? Breaking change vs added functionality? Maybe look at how other emacs packages deal with this?

Also, should we maybe use patch/ for hotfix changes instead of hotfix/ since they are being released as patch version bumps?

[cg505]

So, we don't seem to be completely on the same page here.

My understanding was that all dev work gets merged into master. When we want to do a release, we create a release branch, bump the version, merge it into stable, and that's the release. No work happens on the release branch, because we are releasing the current code (except in the case of patches, where stuff can happen on the hotfix/ branch). So PRs should all be merged into master.

[rye]

This is good, hashing this out is an important step, and if we reach a good versioning scheme here it would be applicable to many other projects.

First,

What is an API change in our context?

I'd say there are two categories: breaking changes where we would need to bump the major version, such as removing old features outright such that a configuration would break if one were to update, and tweaking changes where the backend behavior gets changed but a configuration that properly uses the API would be fine.

I would define the difference between a user "properly" using the API as something like using our pseudo-DSL; if we write kotct/ methods, those are our API, but if someone is extending it or messing with it, that's not considered when we decide whether something is breaking or tweaking.

Maybe look at how other Emacs packages deal with this?

Hopefully we can reach something that semantically uses SemVer without needing to replicate something entirely, but I'd love to hear about other schemes.

---

My understanding was that all dev work gets merged into master.

That is correct. Any new features get put in master. Any fixes of those features also get thrown in master, because master is intended to either be equal to or ahead of stable.

[Out of order] No work happens on the release branch, because we are releasing the current code.

I consider committing version bumps to be "work," but that's besides the point.

[Out of order] When we want to do a release, we create a release branch, bump the version, merge it into stable, and that's the release.

It really seems that this is just a question of merging releases into master or stable first. My philosophy was that master would always be equal to or ahead of stable, thus release/ branches would go into master first and then get merged over to stable. I could see, however, doing the opposite and merging release/ branches to stable first, then merging to master.

In any case, here was my intended scheme:

• Development work takes place on master or on feature branches off of master that get merged into master.
• master and stable do not receive any version bumping commits directly. (nor changelog entry work nor anything of that sort) That all happens on release/vX.Y(.Z)? branches and gets merged into either master or stable. (see above)
  • A release/ branch is created for all planned releases. (This is part of the development process, picking and preparing for a new version.) For instance, if a bug report comes in, we would suggest a version to release this to.
  • This release/ branch can receive a version bump at any point, just as long as it gets a version bump at some point.
  • In the event that this release/ branch is coming from master and contains new features, (constituting a minor release) the branch should be created off of the point where these new features exist, and we name the branch release/vX.Y. (The .0 is implied.) The PR to release this version targets either master or stable, and after the PR is merged the release work will need to get merged into master or stable (whichever the PR was not merged into) to keep the versions in check.
  • In the event of hotfixes, I think hotfixes should generally target specific versions; any work done on these would take place on a hotfix/ branch. To do this properly, we would need to create a release/vX.Y.Z branch for the hotfix to target when the PR is ready. Any additional hotfixes for the same vX.Y version should target the same release/vX.Y.Z branch.

So PRs should all be merged into master.

Yeah.

Since I kinda rewrote this a bunch of times and haven't given it a once-over, please ask away.

[cg505]

So, to be clear, for minor or major version releases, the release/ branch should have master and the commit bumping the version and that's it. Right?

Another thing is that we may want to target multiple minor versions with a hotfix. This is easy enough to do. The hotfix/ branch starts from the latest release tag from the earliest minor version we want to release a new patch version for, and then we create release/ branches for each minor version being targeted. Each merges the hotfix/ branch and the latest release in that minor version.

Other than those things, I agree with this plan.

Finally, my case for merging release PRs into stable:

• If we merge into master the only diff on the PR will be the version bump… this is not very helpful.
• If we merge into stable, the PR will contain everything new for the release.
• master will only be behind stable for the 30 seconds it takes you to merge stable back into master. Then master will once again be ahead of stable.
• If work happens on master between the creation of the release/ branch and the final release, then merging a PR into master and then tagging that merge commit will cause the release to contain those changes even if they aren't meant to be a part of the release. Ideally this doesn't happen but if a release takes a while for some reason (testing or anything like that) then this could potentially happen.

[rye]

So, to be clear, for minor or major version releases, the release/ branch should have master and the commit bumping the version and that's it. Right?

Exactly! 😀

Another thing is that we may want to target multiple minor versions with a hotfix. This is easy enough to do. The hotfix/ branch starts from the latest release tag from the earliest minor version we want to release a new patch version for, and then we create release/ branches for each minor version being targeted. Each merges the hotfix/ branch and the latest release in that minor version.

That's a great idea. I haven't really thought about how to deal with this weird case that actually makes a lot of sense, but that solution sounds like it'll work.

Other than those things, I agree with this plan.

Excellent! 👍

Finally, my case for merging release PRs into stable:

• If we merge into master the only diff on the PR will be the version bump… this is not very helpful.

Fair enough.

• If we merge into stable, the PR will contain everything new for the release.

This is a very good point. Otherwise, it's like we'd want to PR master over to stable.

• master will only be behind stable for the 30 seconds it takes you to merge stable back into master. Then master will once again be ahead of stable.

Good, good.

• If work happens on master between the creation of the release/ branch and the final release, then merging a PR into master and then tagging that merge commit will cause the release to contain those changes even if they aren't meant to be a part of the release. Ideally this doesn't happen but if a release takes a while for some reason (testing or anything like that) then this could potentially happen.

Yeah, that's a weird edge case. If we do perform release candidacy for some reason, this release/ branch will stick around for a while. (We could just bump rc versions on that branch, no need for separate release/ branches in that case.) Hence it does make a lot of sense to develop on master, branch off of master, and then just merge from master over if new stuff needs to get added to the release. Then, stable gets the changes and you resynchronize master. Elegant!

If my reading of your response is correct, let me know and I'll make the changes outlined therein.

[cg505]

So I think we are on the same page now (release PR into stable). If so, I will finish looking at the stuff in the CONTIBUTING.md PRs, and once that stuff is merged, we should release 0.0.0 to test out the process.

[rye]

I would agree. I have re-requested review on #60 now. Make suggestions based on content and stuff—I'll gladly reword and adjust structure.

Then we can release v0.0.0 to verify everything works nicely.

[cg505]

Are we good to close?

[rye]

My initial proposal for this scheme was actually an abridged version of the GitLab Flow idea. @cg505, what do you think of using something more like that flow? Since it's a very good spec I'll leave it as an exercise for you to read it and propose a counter. I'm just not a fan of merging stable back into master and having to use a PR for that. That's silly and a bad use of things; if anything a manual merge from master to stable from the CLI is better.

[cg505]

I agree manual merge is better. GitLab Flow sounds like pretty much what we are doing except that they want to merge bugfixes into master and then cherry-pick into stable... what? They claim it's "upstream first", but clearly stable is upstream from master, so I'm not sure how that works. Seems like it'll make for some nasty rebase shit.

[rye]

Under this scheme, master contains fixes beyond the "stable" release. Versioned releases are thus downstream spare the version bumping commits, because master will be ahead of stable in terms of features. Every bug fix goes into master first and then gets cherry-picked via git-cherry-pick into stable. It's actually pretty clean because only the changes prescribed in that commit get applied. See this doc on cherry-picking.

[rye]

To an extent, I think this works pretty well; VERSION bumps to specific versions are thus kept off master. We can still use release/ branches for this; we'd just not merge back into master.

Another advantage is that when features are being hacked on based on master, the VERSION for those clients is a -pre referencing the next minor release. On stable, the VERSION is clearly going to be something exact that matches the tag, and this gets bumped every time a release/ branch is getting merged (for minor releases).

For instance, compare the VERSION file on gitlab-org/gitlab-ce:master and the VERSION file on gitlab-org/gitlab-ce:9-4-stable: these two are based on master but patches get applied to master, cherry-picked onto X-Y-patch-Z branches, then merged into stable.

This way, only the changes introduced in hotfixes get brought in. If more is needed, more is brought in as well.

I kinda like this. @cg505, should I submit a PR for this new scheme instead? It might make sense for me to spell this out in a step-by-step process instead.

[cg505]

master will be ahead of stable in terms of features

So... stable is upstream.

This looks fine though. It's not my preferred system but it works just as well as any other.

[rye]

Wait, how is stable upstream? Features and bugfixes come from master into stable, so how can master be downstream?

In the end this scheme is just a slightly more semantic way of dealing with hotfixes and such, and I have started revamping the versioning guide. Expect a PR soonish.

[cg505]

https://stackoverflow.com/questions/2739376/definition-of-downstream-and-upstream

When you make changes, you usually want to send them back "upstream"

This is what I was thinking of when I said that. But I also see how you can see stable as downstream in this context. Ultimately it's a semantic difference that doesn't matter. I'm fine with your scheme.

[cg505]

jfc this issue is reaching legendary lengths

[rye]

Yeah, this second pass will probably be the last one though and we'll be able to close. I can't wait to take a bunch of screenshots of this to admire it.