Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VersionBot: Grouping PRs into a single version #16

Closed
lekkas opened this issue Feb 8, 2017 · 51 comments
Closed

VersionBot: Grouping PRs into a single version #16

lekkas opened this issue Feb 8, 2017 · 51 comments

Comments

@lekkas
Copy link
Contributor

lekkas commented Feb 8, 2017

In the first procbot iteration we will basically cut a new version for every merged PR. For PRs that are related, we could use a scheme like:

  1. Add related PRs into a milestone
  2. Proceed with the review/merging process as usual, but only run versionist (i.e. cut a new version and create changelog) after the last milestone PR has been merged.

Actually, we'll have to merge all grouped/milestone PRs just before running versionist. I'm not sure how we can signal process bots to start the process, the first idea that comes to my mind is simply closing the milestone, but we can find something better I guess

@pimterry
Copy link
Contributor

pimterry commented Feb 14, 2017

As discussed on the call:

  • For individual cases this could be a label (add-changelog-line-but-dont-bump-version)
  • For some repos (npm modules especially), we want this behaviour to be the default - opt-in to releases, not opt-out.
    • Normal PRs should just add a line to the changelog, without bumping the version.
    • We could maybe have a new label (create-new-release) that adds a changelog line too, but also groups all outstanding changelog items and creates a release.
    • Might be nice to be able to do this outside of a PR, since that's the workflow here. E.g. open an issue for the new release, discuss there until ready, then @versionbot please release this. when we're done.
    • Milestones could maybe cover that case too. I.e. Normal PRs don't bump versions, but closing a milestone does.

@hedss
Copy link
Contributor

hedss commented Feb 20, 2017

@lekkas @pimterry
I'm tempted to do the following:

  1. Create (or modify) the .procbot.yml file in the root of the repo (see the new resin-io/hq#512 update)
  2. We have a versionbot: version-on-label: true property in there that essentially says 'Don't do any versioning up when dealing with a PR but do ensure every PR has a valid Change-Type: tag`
  3. PRs are opened and Versionbot looks for the version-on-label label either on a line in a commit in the PR or on any related issue:
  • Should it not find one, it doesn't attempt a version up but only merges it once the procbots/versionbot/ready-to-merge label is applied
  • Should it find one, only then does it do a version up before merge

By doing this, it means we can only version up NPM modules when we're satisfied that it's time for a new version. The maintainer creates a new issue milestone which they want the features in, adds the procbots/versionbot/update-version label or similar. This way, as long as every previous PR has had a valid commit message, all the new changes will be in a single version bump.

TL;DR:

  • If the repo config file says not to always version up, then VersionBot only checks to see if there are valid commits.
  • If a PR includes the relevant label (or an issue connected to it does) then the version bump occurs before the merge.

@pimterry
Copy link
Contributor

@hedss Sounds like a sensible way to stop version updates, but I'm not sure about the process to trigger version updates on-demand from there.

I'm a little confused by the detail of some of your description, but it sounds like there's three ways to trigger a version update when a PR is merged:

  1. By including version-on-label within a commit message on the PR
    • This is surprising to me, and is in your description above but isn't mentioned in your TL;DR. Is this correct, or have I got the wrong end of the stick?
  2. By adding procbots/versionbot/update-version to the PR itself before adding ready-to-merge
    • Clear, sensible, great.
  3. By adding procbots/versionbot/update-version to any connected issue before adding ready-to-merge
    • This seems risky to me. It's easy to accidentally connect PRs to issues, especially if there's been lots of discussion with references off to previous issues etc, and this seems like quite a major footgun. Especially later if we link version updates to npm publish etc. Oops!

I think "are we ready to launch a new version" is something that's normally going to be answered either by:

  • Discussion on a ready PR ("Now X/Y/Z is done with this we'll be good to go, so I'll add a label")
  • Out-of-band discussion (in FD: "Anything else we need before release? No? Ok I'll ship it")

Option 2 (labels on PRs) supports the first use case wonderfully. I'm not clear what use cases options 1 and 3 support (it's rare imo that I know a release will should definitely result at the moment I'm committing, before later discussion, or when I file the issue).

I don't think we currently support the 2nd use case (releasing after out-of-band discussion). If I don't have any open PRs then there's no way to trigger versionistbot, but that is a case that comes up semi-frequently I think (because we merge when a PR is complete, not necessarily when we've thought through all the context about what's next). Doesn't mean that needs to block this, since releasing from PRs is the most common case, so we should start there, but it'd be nice to solve that too eventually.

@lurch
Copy link

lurch commented Feb 21, 2017

I guess to satisfy the 2nd use case, there effectively needs to be an "empty PR" with the procbots/versionbot/update-version label applied?
What if there was a standard (not normally present) file you could touch into existence e.g. versionbot_update, which you could then create a PR on, and then if VersionBot sees that file in a PR which has the procbots/versionbot/update-version label, then it'll automatically delete that file in the same commit in which it updates the version?
(Bit of a hacky solution, somebody else will probably come up with something cleaner!)

@hedss
Copy link
Contributor

hedss commented Feb 21, 2017

@pimterry Re-reading my comment, I changed mind half-way through and didn't rewrite the last section. :-/

It was meant to convey that the 'procbots/versionbot/update-version` should either be on the PR or the linked issue. So essentially, 2. from your options (but with added issues as well).

Thinking about it, I think you're absolutely right about not including issues in case something's added by mistake. So, yes, add the label explicitly when you want to version up.

@lurch I really don't like the idea of phantom files. They make me sad. :-(

@pimterry
Copy link
Contributor

Ok, so it's just option 2 from the above then: if you have version-on-label set to false but you do want to do a version update with this merge, then you add the update-version label to the PR before you add ready-to-merge.

That sounds great to me as the first step here. With that in place, this works for the typical npm module workflows (afaik), and we can look at extending it to not require a PR in various ways as a later step. I'm pretty sure we can come up with something not too hacky for that later step, but we don't need to worry about it right now.

@lekkas
Copy link
Contributor Author

lekkas commented Mar 6, 2017

@pimterry @hedss just read this thread (sorry for being late to the party) and this sounds good. So when using the procbots/versionbot/update-version label then procbots/versionbot/ready-to-merge will actually do nothing related to versioning but it will only

  1. Merge the PR
  2. Update CHANGELOG.md, without adding a new version

I also agree that it'd be nice to have a sensical way to signal version bumping (and, soon, publishing?) for npm modules that's not tied to merging a PR and till then I'm also fine riding the PR-merge process to do this

@hedss
Copy link
Contributor

hedss commented Mar 6, 2017

We could in fact have this as a config option now:

procbots:
    githubbot:
        versionbot:
            group-prs: true

(Forget the prop name for the moment, we can 'discuss' that later ;-) ).

So this says 'I wish to, by default, group all my PRs and not run versionist on every merge, simply updating the CHANGELOG with the new details'. I don't believe there's any feature in versionist atm that allows for this. Another reason we probably want to merge into VB. Then by default, a merge will never version bump. The relevant label can be added to a new PR to force a bump.

@lekkas
Copy link
Contributor Author

lekkas commented Mar 6, 2017

@hedss yes, these are exactly my thoughts on what this config option would do (and true i'd suggest a different name for this config option :p but we'll sort this out). We'd then also need to specify the label name that 'triggers' a version up, as it's already discussed in the thread

@hedss
Copy link
Contributor

hedss commented Mar 6, 2017

@lekkas Awesome, in that case I think if we can get the details for versionist together and implemented, the changes to VB should be sorted in about half a day, I think!

@pimterry Does this all sound good to you?

@lurch
Copy link

lurch commented Mar 6, 2017

Just a quick question: there's talk about 'merging' versionist and VersionBot (whatever that means). Will versionist still retain the capability to be run as a standalone tool, independent of everything else?

@hedss
Copy link
Contributor

hedss commented Mar 6, 2017

@lurch It's a good question, there's some back and forth on this. We really need an API VersionBot can hook into and just use it as an NPM module, so we've been mooting the idea of just rolling it wholesale into VersionBot as a utility library.
On the other hand, if people are going to use it as a standalone tool (if we've evidence to support this), then it probably should stay separate codebase.
@lekkas has strong views (I actually don't, for once :) ), so he's best placed to answer this!

@lurch
Copy link

lurch commented Mar 6, 2017

Ahh, thanks. From what you've described above, it sounds like maybe versionist and VersionBot could be two different front-ends to a versionist-lib backend module?

I don't have evidence of versionist being used as a standalone tool outside of resin.io , but I remember that being one of the original intentions of the project ;-)

@hedss
Copy link
Contributor

hedss commented Mar 6, 2017

If we're keeping it as a standalone tool too, I'm happy with just having a single versionist module that can be used either as the tool or pulled in as a library (it's a pretty common NPM model)!

@lurch
Copy link

lurch commented Mar 6, 2017

Cool.

Going back to the original topic of this PR... ;)
One of the things that occurred to me while watching @hedss 's recent VersionBot demo was that he said that every PR adds a message to the Changelog and bumps the version. The intent of this issue #16 is to allow you to group PRs, and only version-bump-on-request.

However, if group-prs: false is set in .procbots.yml (i.e. the existing behaviour) it seems to me that there might be some PRs that you don't want appearing in the Changelog? E.g. if you're fixing a spelling mistake in a code-comment, that probably doesn't warrant a version bump nor a Changelog update, so I was wondering if there could maybe be a way (e.g. via another PR label or maybe a commit-footer-tag) to have versionist / VersionBot still merge a PR (when the appropriate label gets added), but simply "leave it on master" without updating Changelog, until the next PR got merged, at which point Changelog would get updated as normal (and the "new version" would 'transparently' include the previous typo fix).
I realise that's a bit of an edge-case scenario, but the other situation is to either have loads of "typo fix" entries in Changelog, or have people not bother to fix any typos they spot due to the fear of the extra noise it adds to the Changelog?

Am I making sense?? ;-)

@hedss
Copy link
Contributor

hedss commented Mar 7, 2017

I guess we could morph the labels slightly. If procbots/versionbot/no-checks was set, but then a procbots/versionbot/ready-to-merge was set, it'd merge without doing any versioning.

Then again, you could just set the procbots/versionbot/no-checks label at PR creation, and merge manually. Atm, I don't think there's much point just setting the merge label when you know all that's going to happen is it's going to do it for you. Now, if we decide to remove the merge button, that's another matter. :)

TBH, I have no strong opinion either way. Except there's more code to add without a manual merge. ;-)

@lurch
Copy link

lurch commented Mar 7, 2017

If procbots/versionbot/no-checks was set, but then a procbots/versionbot/ready-to-merge was set, it'd merge without doing any versioning.

Hmm, my only concern with having it done via the existing labels, is that it might be too easy to accidentally apply both those labels at the same time, without realising that it means VersionBot will automatically merge without doing a version/changelog update? *shrug*

Am I being overly-concerned about nothing, and people would actually be happy with multiple "typo fix" entries appearing in their Changelogs?

@hedss
Copy link
Contributor

hedss commented Mar 7, 2017

Well if we're grouping PRs, I'm not sure why you wouldn't wait till the next grouping to fix these typos anyway? Or patches that don't need a rollout now. I thought that was part of the point of grouping in the first place?

@lurch
Copy link

lurch commented Mar 7, 2017

I guess in hindsight I'm specifically talking about changelog-updating (i.e. should there be a mechanism for merging in typo fixes without "typo fix" entries appearing in the changelog?), which is subtly different from version-updating. Sorry for confusing everybody - let me know if that ought to be split out to a separate issue.

@pimterry
Copy link
Contributor

pimterry commented Mar 7, 2017

Yeah, I can see the use case for this, and that makes sense. Grouping PRs is a change to disable version releases for some merges, this is a change to disable version releases and changelog updates for some merges.

Sounds sensible and fairly easy to do imo though. Another label is an ok solution, but an even more interesting one would be a different change type, e.g. change-type: none, which marks something as being so small and irrelevant and non-behaviour changing that it doesn't warrant any changelog or version bump at all, but tells versionistbot that it's not just that you've forgotten to include a change-type, you're doing this on purpose.

Or of course you could just use no-checks, merge the PR by hand, and ignore versionbot entirely 😄

Separate issue for discussion on this one is probably sensible 👍.

@lurch
Copy link

lurch commented Mar 7, 2017

but an even more interesting one would be a different change type, e.g. change-type: none

Hah, I was already considering suggesting exactly that (i.e. a change-type level 'below' patch), but I didn't know if it'd upset the SemVer purists ;-)

Will create a new issue in the morning 👍.

@hedss
Copy link
Contributor

hedss commented Mar 8, 2017

I want to bring @alexandrosm and @lekkas back into this, as there's now two different trains of thoughts on grouping and always bumping.

@hedss
Copy link
Contributor

hedss commented Mar 8, 2017

@lurch Any change-type: none issue needs to go into versionist, as that's ultimately what does this, and should be doing it once it has a suitable API to allow VB to 'just call it'.

@lurch
Copy link

lurch commented Mar 8, 2017

So, with that little side-issue now split off to it's own discussion, I still wonder if, when group-prs: false is set in .procbots.yml, there should be a label you can apply to a PR to signify 'merge this PR (and update the Changelog) but don't bump the version just yet'?

Also, if group-prs: true is set in .procbots.yml and the procbots/versionbot/update-version label is set on a PR (to indicate that a version-bump is now required), would it be useful for VersionBot to produce a GitHub link that could be clicked on to view all the differences (or perhaps just the differences in the Changelog) since the last version-bump?

@alexandrosm
Copy link

alexandrosm commented Mar 8, 2017 via email

@pimterry
Copy link
Contributor

Hmm, ok, I think being able to batch patch & minor updates as well makes for a nicer model. We should talk about this in the next process call.

My main issue is that doing this piecemeal makes the development flow really strange:

  • I put in a PR with a major change type.
  • Unlike any other PR, here it doesn't result in a version bump, which will be potentially surprising if I haven't made other major bumping changes recently, which should be common.
  • The next person who puts in any other patch/minor PR before this major version is released now automatically creates a major version bump, which will be potentially very surprising if they've just made a typo fix.

You could have some kind of 'version frozen' state you flip into on a non-bumped major change, which you have some process to get out of later, and which disables all versioning in the meantime. I think there is a route through there, but it all gets a bit complicated.

On the background too though: I can't talk for versionist, but semver doesn't have much to say, in spirit or otherwise, about doing more small rapid-fire bumps, and quite a bit of the spec actively encourages thoughtful & carefully managed bumping instead. For example, the downsides of doing major releases in quick succession is explicitly in there. That isn't a concession against semver, it's a more or less a recommendation from the spec itself.

I definitely do agree we should avoid creating lots of versions for purely aesthetic reasons. Npm modules are different to live components though, as there's much more practical effect from a version bump. Repeated major version bumps clearly make real pains, but even for non-major versions it's inconvenient to downstream users if we create many many minor and patch versions. Their builds and dev environments become less reproducible (the output changes each time we do a release - frequent irrelevant releases make this messy), our changelogs gain noise and get harder to follow, and as a person it's harder to understand what the contents of a release is (rather than a series of related features/bug fixes arriving together as a coherent whole, they come out in a series of separate releases).

Anyway, we should have a call to cover this properly. I'll ping in flowdock.

@lurch
Copy link

lurch commented Mar 10, 2017

Not sure how much of it applies to the discussion here, but I stumbled across this the other day:
http://keepachangelog.com/

@pimterry
Copy link
Contributor

Once a PR has been merged, doesn't it just get seen as a bunch of separate commits in git? So in theory there's no difference between five PRs each containing two commits, and a single PR containing ten commits?

Yes, that's the plan, and if that's the case, I think this works great.

If you send many PRs to a branch, instead of master, and then merge that branch into master later, they should get treated as if it was just a single PR to master with all those changes included (separate changelog lines for each change, but a single version bump). I haven't tested this yet, but I will shortly. Is there some reason that doesn't solve this issue?

Might also be nice if VB could update the changelog in the non-master branch, so you can see what's been done, before merging that branch back to master?

Yeah, that bit's tricky, because you need to start merging created changelog files, not just parsed entries from the commits. Probably doable, but I'm pretty sure it doesn't work quite yet. You'd also need to be able to add changelog entries without bumping the version anyway (as in the original plan on this ticket), or to have some kind of version squashing or something... More complicated either way.

I definitely agree though that this info (what's about to be added to the changelog) would be helpful, but that's true on normal PRs too. I've filed a separate ticket for that part: #67

@lurch
Copy link

lurch commented Mar 16, 2017

Is there some reason that doesn't solve this issue?

If it works, sounds to me like it's a good workflow.

because you need to start merging created changelog files, not just parsed entries from the commits.

Exactly, that's why I suggested that changes made to changelog on a branch, should be ignored when merging that branch back into master. IYSWIM! So the master changelog would only ever be updated by versionist parsing the commit history.
But I dunno if it's possible to exclude specific files when merging one branch into another?

@pimterry
Copy link
Contributor

Exactly, that's why I suggested that changes made to changelog on a branch, should be ignored when merging that branch back into master. IYSWIM! So the master changelog would only ever be updated by versionist parsing the commit history.
But I dunno if it's possible to exclude specific files when merging one branch into another?

Ah, ok, I can see how you could do that kind of thing. You don't need to do ignore it file by file though, you could do it by commit author. If versionbot filters out its own commits before merging, you'll get the expected result. I'd rather this was surfaced in the github ui though, simply because doing magic to add and then automatically remove things from the git history sounds fiddly and error-prone. Anyway - this should be on #67, not here.

@lekkas
Copy link
Contributor Author

lekkas commented Mar 16, 2017

@pimterry SGTM

@lekkas lekkas modified the milestone: next Mar 21, 2017
@lekkas
Copy link
Contributor Author

lekkas commented Mar 22, 2017

@pimterry @lurch I just reread the thread, I'm strongly against committing a changelog file in the release branch that will be ignored when the release branch gets merged onto master, for the reasons @pimterry stated

I'd rather this was surfaced in the github ui though, simply because doing magic to add and then automatically remove things from the git history sounds fiddly and error-prone.

So I'd rather we move forward with the plan Tim nicely summarised here and update VersionBot to

@hedss how does this sound to you?

@lurch
Copy link

lurch commented Mar 22, 2017

I'm strongly against committing a changelog file in the release branch that will be ignored when the release branch gets merged onto master

Fair enough. I still think it might be nice to be able to have some kind of 'preview' of what additions a release branch would add to the master Changelog, should that release branch get merged back to master. I'll add some more comments to #67

@lurch
Copy link

lurch commented Mar 22, 2017

I dunno if it's something that the github interface already enforces, but presumably we shouldn't allow a release branch to be PRed back to master if that release branch still has open PRs against it?

@hedss hedss added this to Backlog in ProcBot Work Mar 22, 2017
@hedss hedss changed the title Grouping PRs into a single version VersionBot: Grouping PRs into a single version Mar 22, 2017
@hedss hedss moved this from Backlog to High Priority in ProcBot Work Mar 22, 2017
@hedss
Copy link
Contributor

hedss commented Apr 11, 2017

@alexandrosm
Copy link

@hedss are we closing this?

@hedss
Copy link
Contributor

hedss commented Jun 16, 2017

Yes. Yes we are.

@hedss hedss closed this as completed Jun 16, 2017
@hedss hedss moved this from High Priority to Closed in ProcBot Work Jun 16, 2017
@lekkas
Copy link
Contributor Author

lekkas commented Jun 21, 2017

@hedss what was the final resolution of this? Do we need to update any process docs?

@hedss
Copy link
Contributor

hedss commented Jun 21, 2017

@lekkas I don't believe so, I don't think we ever said you'd be able to group PRs in the Commit/PR doc. (having scanned it, can't find mention of such).

@lekkas
Copy link
Contributor Author

lekkas commented Jun 21, 2017

OK , just to clarify because this is a recurring question - We have mentioned this alternative path in the past:

a. Create a temporary next branch
b. Use change-type tags and merge (manually, with the GH button) PRs there
c. When ready to cut a new version, create a PR to merge next onto master

Is this a workflow supported by VersionBot?

In any case, do we abandon the concept of grouping PRs to a single version at all cc @alexandrosm ? Asking because I'd love VersionBot in resin-cli for instance (sent a few PRs today, got a hard reminder of how inconvenient manual changelog entries are) but then the per-PR versioning scheme will essentially look like a regression in its current Changelog format (e.g. see https://github.com/resin-io/resin-cli/blob/master/CHANGELOG.md#5110---2017-06-19)

@hedss
Copy link
Contributor

hedss commented Jun 21, 2017

@lekkas Yes, that would work because only master is protected, so as long as the base (in this case next) is a different branch it won't matter.

@lurch
Copy link

lurch commented Jun 21, 2017

But does a branch being non-protected (i.e. not master) mean that VersionBot won't operate on it, or does the project admin need to remember to never apply any VersionBot labels to PRs on non-protected branches, and instead merge them manually instead? With a repo having some PRs against master and other PRs against non-master branches, I can see that it'd be easy for a busy maintainer to get confused and accidentally apply a VersionBot label to a non-master branch (which would then do a version-bump, which is what we're trying to avoid)... ?

@hedss
Copy link
Contributor

hedss commented Jun 22, 2017

VersionBot would still operate on the branch, yes. The events are per-repo, not per-branch. And yes, if a maintainer added the label to the branch that wasn't meant to be doing the bump, then you'd get a bump. I guess one way round this would be to have a new property in the repository.yml (replacing .procbots.yml for repo specifics) that lists only base branches where a version bump should be applied on merge (so say: bump_base_branches: [ 'master' ], yeah the name sucks). This would mean that whilst checks would still be carried out on all branches (which is what you'd want, still need valid commit messages), the ready-to-merge label would skip version bumping on any merge not going to a specified base branch.

Thoughts?

@lurch
Copy link

lurch commented Jun 22, 2017

Apart from the name, SGTM ;-)
And presumably if there's no bump_base_branches specified then VB would operate on all branches, the same as it does now?
And if bump_base_branches is being used, VB should probably emit a warning comment explaining why it didn't bump the branch if the ready-to-merge label gets applied to a PR on a different branch? (to help prevent the maintainer getting confused, and possibly also acting as a reminder that the branch being operated on will still need to be merged to one of the bump_base_branches before the version will get bumped?)

It's a bit early in the morning for me to think about it too hard, but in which circumstances would you want bump_base_branches to be set to more than one branch? Wouldn't two (or more) bump_base_branches mean that you might end up with the same version-label on two separate branches? (i.e. two different versions of the code having the same semver version)

@hedss
Copy link
Contributor

hedss commented Jun 22, 2017

Yes, we could warn (though presumably the maintainer would have set it up in the first place). You wouldn't neccessarily end up with two branches of the same version. Imagine a policy where all external contributions get merged into a holding branch with no version bumping, and all internal contributions get merged into another holding branch which is bumped. Finally you want to merge both branches together onto master, you merge them into another branch which is then merged to master. You get a final version bump which essentially versions all of the externals in one lump, thus creating your now publishable version. Maybe it's a bit far fetched. :)

@lurch
Copy link

lurch commented Jun 22, 2017

Finally you want to merge both branches together onto master, you merge them into another branch which is then merged to master.

Totally off-topic, but could that also be done by cherry-picking commits from both branches into a single PR against master? (I've never gone that complex with git!)

@hedss
Copy link
Contributor

hedss commented Jun 22, 2017

Yeah, it def. could, but it'd be a bit more work. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

5 participants