Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release Process #2490

Merged
merged 50 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
11aa7ac
Update README
ggwpez Nov 24, 2023
5eea0df
Add RELEASE doc
ggwpez Nov 24, 2023
d8b7524
Update README
ggwpez Nov 24, 2023
d622a7c
Add AUDIT.md
ggwpez Nov 24, 2023
1e5b3b0
Typo
ggwpez Nov 24, 2023
becbb79
Fix version format
ggwpez Nov 24, 2023
316579d
Update README.md
ggwpez Nov 27, 2023
11b77b0
Apply suggestions from code review
ggwpez Nov 27, 2023
6cdfd59
Remove leeway
ggwpez Nov 27, 2023
35d87bf
Add sections about Testnets and versioning
ggwpez Nov 27, 2023
5189616
Only publish changed crates
ggwpez Nov 27, 2023
a769906
Explain more
ggwpez Dec 1, 2023
6d61040
Apply suggestions from code review
ggwpez Dec 4, 2023
d9bf9ef
Update docs/RELEASE.md
ggwpez Dec 8, 2023
71d9062
Update docs/RELEASE.md
ggwpez Dec 8, 2023
4d65917
Explain more stuff
ggwpez Dec 8, 2023
1cc2780
Update docs/RELEASE.md
ggwpez Dec 8, 2023
423d0ab
Review fixes
ggwpez Dec 8, 2023
9d7bdfd
Merge remote-tracking branch 'origin/oty-release-doc' into oty-releas…
ggwpez Dec 8, 2023
2bdd380
No GH release for nightly
ggwpez Dec 8, 2023
51322b3
SemVer exempt node internal crates
ggwpez Dec 8, 2023
2cd25d9
Exclude unstable APIs
ggwpez Jan 2, 2024
554b247
Minor fixes
ggwpez Jan 2, 2024
d11e585
Improve clobbering script
ggwpez Jan 2, 2024
26d5a8a
Merge remote-tracking branch 'origin/master' into oty-release-doc
ggwpez Jan 8, 2024
7de5e5e
Update release doc
ggwpez Jan 8, 2024
ad657d9
newlines
ggwpez Jan 10, 2024
7158244
Update docs/RELEASE.md
ggwpez Jan 15, 2024
bfad729
Apply suggestions from code review
ggwpez Jan 15, 2024
1e10077
Update README.md
ggwpez Jan 23, 2024
f8dea0b
Update README.md
ggwpez Jan 23, 2024
3341c6b
Update docs/AUDIT.md
ggwpez Jan 23, 2024
3702afb
Update docs/AUDIT.md
ggwpez Jan 23, 2024
dce630d
Update docs/RELEASE.md
ggwpez Jan 23, 2024
c3014c4
Update docs/RELEASE.md
ggwpez Jan 23, 2024
702118b
Update docs/RELEASE.md
ggwpez Jan 23, 2024
ae7e73c
Apply suggestions from code review
ggwpez Jan 23, 2024
7a4b801
Apply suggestions from code review
ggwpez Jan 23, 2024
6169ee8
Mainline -> Stable
ggwpez Jan 23, 2024
d03a2d3
Use uniform numbers
ggwpez Jan 23, 2024
80b81b1
Spelling
ggwpez Jan 23, 2024
9328f76
Wrap lines at 120
ggwpez Jan 23, 2024
e7032c5
Condense bump hint
ggwpez Jan 23, 2024
818c013
Cleanup
ggwpez Jan 23, 2024
f36af66
Merge branch 'master' into oty-release-doc
ggwpez Jan 23, 2024
d01c07a
Update Westend to nightly every 2w
ggwpez Jan 23, 2024
ef1f428
Add hint to Fellowship releases
ggwpez Jan 23, 2024
d11235f
Line breaks
ggwpez Jan 23, 2024
09e539a
Update README.md
ggwpez Jan 24, 2024
fc28e27
MR -> PR
ggwpez Jan 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,21 @@ Polkadot.

Cumulus is a set of tools for writing Substrate-based Polkadot parachains.

## Releases

> [!NOTE]
> Our release process is still Work-In-Progress and may not yet reflect the aspired outline here.

The Polkadot-SDK has two release channels: `mainline` and `nightly`. Production software is advised to only use `mainline`. `nightly` is meant for tinkers to try out the latest features. The detailed release process is described in [RELEASE.md](docs/RELEASE.md)
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

### Mainline

`mainline` releases have a support duration of **three months**. In this period, the release will not have any breaking changes but only receive bug- and security fixed on a **two week** cadence.
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

### Nightly

`nightly` releases are released every night with possibly breaking changes and offer **no SemVer** guarantee between each other. They have pre-releases version numbers in the format `major.0.0-nightlyYYMMDD`.
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

## Upstream Dependencies

Below are the primary upstream dependencies utilized in this project:
Expand Down
17 changes: 17 additions & 0 deletions docs/AUDIT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Audit

Audits are conducted to ensure the absence of severe or exploitable bugs. MRs are generally merged without audit into the `master` branch. The `audited` tag is used to track the latest audited commit of the `master` branch. This means that audits need to happen in order of being merged.
athei marked this conversation as resolved.
Show resolved Hide resolved
This is an optimistic approach that lets us develop with greater speed, while requiring (possibly) large refactors in the failure case.

Audits can be deferred if the logic is gated by an `experimental` feature or marked as "Not Production Ready" within the first line of doc. Such changes should be queued manually before these warnings are removed.

## General Guidelines for what to Audit

There is no single one-fits-all rule. Generally we should audit important logic that could immediately be used on production networks. If in doubt; ask in chat or in the Merge Request.
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

## Requesting an Audit
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

1. Add the PR to the project `Security Audit (PRs) - SRLabs`
1. Set status to Backlog
1. Assign priority, considering the universe of PRs currently in the backlog, or just leave it as TBD
1. Add the component
ggwpez marked this conversation as resolved.
Show resolved Hide resolved
61 changes: 61 additions & 0 deletions docs/RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Processes

The following processes are necessary to actualize our releases. Each process has a *Cadence* on which it must execute and an *Responsible* that is responsible for autonomously doing so and reporting back any error in the RelEng<sup>1</sup> channel.
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

## Mainline Release

Cadence: every two weeks. Responsible: Release Team.

This process aims to release the `release` branch as a *Mainline* release every two weeks. It needs to start three days before the proposed release deadline to allow leeway for unexpected issues. This process should eventually be automated.
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

### Steps

1. [ ] Check if process [Clobbering](#clobbering) needs to happen and do so first, if that is the case.
1. [ ] Check out the latest commit of `release`.
2. [ ] Verify all CI checks of that commit.
3. [ ] Announce that commit as cutoff *Mainline* for a release in the General<sup>2</sup> chat.
ggwpez marked this conversation as resolved.
Show resolved Hide resolved
4. [ ] Bump the semver of all crates <!-- FAIL-CI: We need some better process here on how to do it exactly -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be really not done. IMO we should bump the crate versions as we merge stuff to unstable.

I had written here some things: #2366

I mean I know that is not a perfect thing, but probably the best we can do. IMO this should not be done by a single person or after the fact as this is otherwise too hard to follow.

Copy link
Member Author

@ggwpez ggwpez Nov 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we bump the version of each crate (and all its transitive dependants) in each MR, then we will have huge merge conflicts.
I think if we instead note the version bump in the prdoc then it can happen automatically at release process.

It would also require is to backport the version bumps from crates-io, leading to even more conflicts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also require is to backport the version bumps from crates-io, leading to even more conflicts.

Hmm? Why would we need to backport version bumps? I mean the version isn't bumping automatically.

I think if we instead note the version bump in the prdoc then it can happen automatically at release process.

If you change something deep inside the tree, I don't want to copy past 300 crate names and their bumps to a prdoc file...

But if we bump the version of each crate (and all its transitive dependants) in each MR, then we will have huge merge conflicts.

We should not bump in every pr. Bumps should only be done as they are required between releases.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm? Why would we need to backport version bumps? I mean the version isn't bumping automatically.

Yea I see. I thought about using prdoc to not the semver bump of each crate that i changed, and a script can then take the maximum of all bumps and apply them.

If you change something deep inside the tree, I don't want to copy past 300 crate names and their bumps to a prdoc file...

It could suffice to note the changed crates - all dependant ones could bump automatically.

We should not bump in every pr. Bumps should only be done as they are required between releases.

Okay yea i read it in the other MR from you. So:

  • developer changes one crate and bumps that.
  • CI reminds him to not bump multiple times but to also bump all dependant crates
  • CI checks that there are no obvious and unintended breaking changes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could suffice to note the changed crates - all dependant ones could bump automatically.

Bumping the dependent crates is not that easy. I mean you only need to bump the dependent crates if you re-export anything as public api.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But when we change something in sp-io for example, without modifying the public API but some implementation detail, then we should bump all dependants, or?
I am not sure if its easy to manually select which crates to bump, since there is the chance of missing some. If we always bump all dependants then maybe we bump too many, but at least we never miss any.

Copy link
Member

@liamaharon liamaharon Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not bump in every pr. Bumps should only be done as they are required between releases.

Assuming we are talking about the stable branch here, I disagree for two reasons

  1. It will be much easier for the developer who made each PR to confidently know which crates changed and how, and for them to make targeted bumps in their PR.

    Otherwise, release team will need to do a massive effort identifying and bumping crates based on changes of many PRs from a long time ago they don't understand. This I feel is doomed to fail and is unfair to the release team.

    Also, the developer will be expected to write a PRDoc right? If they have the knowledge to write a PRDoc, they may as well bump appropriate crates while they're at it. Otherwise it's just leaving unnecessary extra work for later.

  2. We should strive for automated releases when code is merged to master, no more manual release business. This requires the developer to bump their crates when they merge to to master.

But when we change something in sp-io for example, without modifying the public API but some implementation detail, then we should bump all dependants, or?

Yes, I think we should. There are tools to automate these common workspace release tasks: https://github.com/Byron/cargo-smart-release https://github.com/crate-ci/cargo-release https://github.com/pksunkara/cargo-workspaces

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it now to make the developer update the crates versions in their Merge Requests (where necessary).

Copy link
Member

@liamaharon liamaharon Dec 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI working on a tool for this https://github.com/liamaharon/cargo-workspace-version-tools

Currently able to sync (catch up) our Cargo.toml versions with crates.io. Next will add functionality to bump a crate and automatically have dependent crates also bumped.

5. [ ] Abort the release process and announce so in General if there are no bumps needed.
6. [ ] Create a merge request to `release` with the proposed SemVer bumps.
7. [ ] Announce this merge request in the *General* channel to quickly gather reviews.
8. [ ] Merge it into `release`.
9. [ ] Verify all CI checks.
10. [ ] Announce the intent to do a *Mainline* release from the resulting commit hash in RelEng.
11. [ ] <!-- The release team has internal checklists for QA i think, should we mention this? -->
12. [ ] Release all crates to crates.io using [parity-publish](https://github.com/paritytech/parity-publish).
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

## Nightly Release

Cadence: every day at 00:00 UTC+1. Responsible: Release Team
ggwpez marked this conversation as resolved.
Show resolved Hide resolved

This process aims to release the `master` branch as a *Nightly* release every day. The process can start at 00:00 UTC+1 and should automatically do the following steps.

1. [ ] Check out the latest commit of branch `master`.
3. [ ] Compare this commit to the latest `nightly*` tag. Announce that the process was aborted in the RelEng chat since there were no changes.
4. [ ] Verify all CI checks of that commit.
5. [ ] Set the version of all crate to `major.0.0-nightlyYYMMDD` where `major` is the last released `major` version of that crate plus one.
6. [ ] Tag this commit as `nightlyYYMMDD`.
7. [ ] Announce the intent to do a *Nightly* release from that tag in the RelEng chat.
8. [ ] Release all crates to crates.io using [parity-publish](https://github.com/paritytech/parity-publish). <!-- FAIL-CI: I think Morgan fixed that tool so it would only release crates that had changes, or that had one of their transitive dependencies changes. That would help, since otherwise we always push 400 crates or so. -->

## Clobbering

Cadence: every 6th release (~3 months). Responsible: Release Team

This process aims to bring branch `release` in sync with the latest audited commit of `master`. It is not done via a Merge Request but rather by just copying files. It should be automated.

The following script is provided to do the clobbering.

```bash
git fetch
git checkout release
git reset --hard origin/audited
git push --force release
```
ggwpez marked this conversation as resolved.
Show resolved Hide resolved



# Footnotes

1: `RelEng`: The *RelEng: Polkadot Release Coordination* Matrix channel.
2: `General`: The *General* Matrix channel.
Loading