Proper release cycle for 0.28+

[dr-orlovsky]

In order to avoid issues like we had with the latest secp256k1 release I propose to do a series of rust-bitcoin RC releases before releasing 0.28.0 (and all future major versions).

Why this is required and we can't just use github master branch from a dependencies to check before the release? One of the main reasons is because there are multiple projects not just depending on rust-bitcoin, but also depending on other crates, which expose APIs using rust-bitcoin. At least, this includes miniscript, electrum-client and many others, so every crate using them can't just switch to github master for rust-bitcoin.

So, the proposal is to have rust-bitcoin 0.28.0-rc.1 first, than do the same for miniscript - and let existing projects using both of them to test these RCs for a while.

[Kixunil]

I think cargo doesn't select RC versions by default (or was it cargo add?) so maybe the difference is not that big but what you're proposing seems like an improvement over current situation.

[dr-orlovsky]

I mean that downstreams will manually use bitcoin = "0.28.0-rc.1" and miniscript = "7.0.0-rc.1" (and electrum-client, once it is also updated, and so on) and be able to test the build.

I have many layers of crates using both (for instance in LNP lightning node implementaiton, descriptor wallet and RGB) and have to do a lot of testing. Adding custom branches/commit github references everywhere is not that efficient.

[Kixunil]

Isn't it the same amount of work? You have to change a version in each crate or you have to change it to git declaration.
It looks like this is a job for an automated tool.

[dr-orlovsky]

No, it is not just about amount of work: this is about third-party repos which you can't change. If they don't do rc release (miniscript etc), there is no way for you to build your repo with the new rust-bitcoin other then to do a fork each of them and set them to use rust-bitcoin from a github reference.

They may do a custom branch/tag, but this will be much less standard and hard to coordinate - plus can change anytime, unlike the RC version released to crates.io.

[Kixunil]

Well, you then need to convince each project to release RC with version bumped (and all API breaks fixed) and wait for them to do so. So it's roughly same amount of work just distributed between multiple people. :)

[dr-orlovsky]

Not the same amount of work: there is no need to convince miniscript (the same people as rust-bitcoin) and for electrum-client they are usually doing this anyway (also @RCasatta here can do that since he is a part of that project)

[apoelstra]

concept ACK. Yes, it would be good to have RCs that our own projects can explicitly opt into, because then we would catch "stupid" mistakes involving new types that don't have derives, functions that return types that aren't actually exported, macros that don't work when called from dependent crates, etc, etc.

It would be nice if others would also use the rc's, but I don't think it's needed or expected.

cc @tcharding

[romanz]

concept ACK for electrs

[tcharding]

I think we really are asking too much of downstream projects, I don't see why downstream should have to beta test our releases. Testing rc releases is the job of rust-bitcoin devs IMO. Another solution would be to come up with a set of projects that gives us good enough coverage and then fork those someplace (all into a single repo?) that we can all see, then do the rc testing there. That way the work is not duplicated and everyone can see the PRs to see whats involved with updating without having to do it themselves. Sounds like a suitable boring task for someone who is on an open source grant to do ...

A few more random points:

• This method would probably work best with branches not crates.io releases
• This would, as an artifact, provide patches for those projects to use when/if they want to upgrade

[apoelstra]

@tcharding we have several downstream projects (rust-elements, rust-miniscript, rust-lightning, electrs) which are maintained by the rust-bitcoin developers.

The idea behind using rcs on crates.io is that it'd be more convenient, but functionally the same, as using git branches.

[tcharding]

Got it, cheers.

[tcharding]

Reporting back after some initial testing. Using crates.io is not going to work because any bugs one finds that break the build of downstream projects have to be patched, this implies using branches either pushed to github or locally using path.

Perhaps this flow will work?

1. Tag a commit with rust-bitcoin-0.28.0-rc.1
2. devs checkout this branch for testing then checkout their own local branch for patching (e.g. rust-bitcoin-0.28.0-rc.1-test)
3. Raise PRs against master as usual
4. Every x hours/days/patches do a new tagged release candidate
5. Once it stabilizes tag and do the real release on crates.io

[dr-orlovsky]

@tcharding not sure why this not going to work; I've been using this for my crates (I have >50 of libs to manage) for years and it worked well.

You start with 0.28.0-beta.1 and do then 0.28.0-beta.N as patches, then move to 0.28.0-rc.1 once all bugs are fixed, have a few RCs more if needed and then do a full 0.28.0 release.

[apoelstra]

I agree with @dr-orlovsky -- it is true that when users are testing patches locally they have to mess around with local paths or git branches, but the idea is that we'd be in "rc mode" only for a short time, and have quick turnaround pushing new rcs with new bugfixes.

[tcharding]

I've not fully internalised your position but I'm happy to defer to you guys on this, I'm not particularly experienced in this are. One question, this method implies that every project in the stack is ready to do a release or were you imagining that some projects might leave the rc release up for a longish time?

[apoelstra]

No, ideally rcs' would only be out for a few days, while downstream projects do sanity checks and try to use the new APIs.

[tcharding]

We have a 'feature freeze' while rc releases are active, right? I.e no PRs merged except bug fixes?

[tcharding]

In order to try and assist those trying to merge patches that are needed during the release candidate cycle I added a label RC fix. Please remove/rename/recolor as you see fit.

[Kixunil]

I think we should. Also good idea with the label.

[apoelstra]

Thanks! Agreed.

Though there is a bit of a blurry line -- if we add e.g. a forgotten #derive, is that a "new feature"?

[tcharding]

Yeah I was wondering about this sort of thing. I really don't want to be the guy that puts a broken patch into the final rc release that doesn't get that heavily tested because its so late in the cycle and then gets into the non-rc release. Perhaps this gives us the answer to that question, if its early in the rc cycle then it counts as a bug fix and if its nearing the end of the rc cycle be more conservative?

[Kixunil]

I think we should weight in the scope too. Adding an uncontroversial derive or a simple access method should be fine. Adding a whole module with a bunch of types and methods is not.

[tcharding]

Are we doing another RC release, its been a while on rc-1 now?

[apoelstra]

@tcharding I would like to get a rust-secp major release out, and then I think we can do a major release here. Or at least another RC. I have spent 4 or 5 weeks now begging to get PRs acked by maintainers but there are not many on that project.