Mockito Continuous Delivery Pipeline 2.0

[mockitoguy]

Proposed changes on easy-to-grasp diagrams are documented on Mockito blog article.

Current release model

Every code change results in a new version in Central repository (not snapshot, a real version). More information on Continuous Delivery in Mockito.

Given high rate of new versions, the community and current tooling is not comfortable in taking in new Mockito versions at fast pace. More feedback and the discussion in issue #618.

Proposed changes

Release every change but not all releases go to Central. Push to standard repository (JCenter / Maven Central) only substantial releases. Push remaining versions to less prominent, but still public space for early adopters.

Glossary: What are major, minor, patch versions? Explanatory example: Version 2.6.5 has major=2, minor=6, patch=5.

1. Every merged pull request produces new version that lands in Bintray "mockito/maven" repo.
2. Some versions we produce are "notable versions", by default those are new major (3.0.0, 4.0.0) and new minor versions (3.1.0, 3.2.0). In contrast to regular new versions, the notable versions are included in well-known central repositories: JCenter and Maven Central.

For more details, see the wiki page paragraph on continuous delivery.

Action list

• - Configure 2.7.22 as last notable version because it was the last version published to central
• Release notes:
  • - detailed release notes link to notable release notes and vice versa
  • - release notes inform where the jars can be found (Release notes indicate target repository shipkit#4)
  • - link to wiki documentation
• - Publish next notable version
• - procedure of promoting to notable release is documented - https://github.com/mockito/mockito/blob/release/2.x/README.md#how-to-release-new-version
• - the new release model is comprehensively documented - https://github.com/mockito/mockito/wiki/Continuous-Delivery-Details#cd

Optional:

• - release notes have clean format (Concise and readable release notes format shipkit#3)
• - notable release (target repo) can be forced by commit message

Implementation details

• https://github.com/mockito/mockito/wiki/Continuous-Delivery-Details#cd
• https://github.com/mockito/mockito/blob/release/2.x/README.md#how-to-release-new-version

[TimvdLippe]

Every minor or major version change lands in both

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units). Therefore I would say to not do it automatically, but presumably faster than with patch releases. E.g. minor every week published, patch every month.

Example: “community” repo has 2.8.1, 2.8.2, 2.8.3, at the end of the month, 2.8.3 is promoted to production.

This means that 2.81 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

I'm not completely happy with repository naming "community" and "production"

Maybe trunk and production?

New version is published under following conditions

I would like to add an override here for to be able to push a new version. E.g. not pushing an existing version to production, but publishing a new version without manually editing javadoc to artificially trigger the release mechanism

Overall I agree with the proposal, just some small comments. Glad that we are taking action in this direction 👍

[mockitoguy]

Great feedback thank you!

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units).

"minor" version (e.g. middle number :) is not automatically bumped by the system. It's us (or the PR submitter) that makes a judgement call to explicitly bump minor version. This decision is based on the gravity of the changes. We don't bump minor version unless it has substantial changes. If it does have substantial changes, it probably should be released to mass audience. At least this is my reasoning. If we don't automatically publish minor versions we can end up with versioning scheme where "community" and "prod" repo have pretty much random versions. If we publish minor versions automatically, "prod" contains announcable, significant versions that add substantial value, e.g. 3.1.0, 3.2.0, 3.3.0 and occasional patch release like 3.3.1. "community" repo would have all changes 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, etc. Thoughts?

This means that 2.8.1 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

The idea is to only include substantial versions in official repo. If the user wants to bisect, he needs configure "community" repo explicitly in pom / build.gradle. In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

Maybe trunk and production?

Let's keep thinking. Some other ideas: all-versions/announcable-versions, all-artifacts/significant-artifacts, early-adopters/slow-adopters.

Given that "minor" version (e.g. middle one) is updated explicitly by the PR submitter, he controls

I would like to add an override here for to be able to push a new version.

Good idea. I will add it to the design.

Really nice feedback. Thanks! We can tweak/iterate the process as we learn more, too.

[TimvdLippe]

We don't bump minor version unless it has substantial changes.

Makes sense, 👍

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

Let's keep thinking.

Latest - Stable?

[arend-von-reinersdorff]

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

I can think of two popular projects which sometines skip versions:

• Maven, missing versions eg 3.3.2, 3.3.4-3.3.8 (and upcoming 3.5.0 will skip 3.4.0)
• Tomcat, missing versions eg 8.5.10, 8.5.7, 8.0.40

[mockitoguy]

@arend-von-reinersdorff, thank you for feedback and examples!
@TimvdLippe, currently, I most like following naming scheme for repositories: "all-versions" and "notable-versions".

[tinkerware]

This looks good! I agree with the model where minor versions are in release channel (notable-versions) and patch versions are in beta channel (all-versions), modulo hot fixes. If patch versions are pushed to notable-versions at the time they are released, you defeat the purpose of the release channel; your "notable versions" channel would churn multiple times daily again.

Maybe Mockito can push all interleaved patch versions available only on the beta channel to the release channel when the release channel is updated? For example:

Before:

release channel – 2.3.0
beta – 2.3.0, 2.3.1, 2.3.2

After:

release channel – 2.3.0, 2.3.1, 2.3.2, 2.3.3 (hotfix)
beta – 2.3.0, 2.3.1, 2.3.2, 2.3.3

This may go against the principle of least surprise, since someone browsing the version history of release channel will notice that suddenly not one but three new versions became visible. This may be acceptable if bisecting and keeping full version history on the release channel are important for the project. As I said before, I don't have a strong preference for this model, except to say that patch versions should not appear in the release channel at the time of their release unless they are hot fixes.

[mattnelson]

I have an alternative opinion from everything proposed so far. I propose introducing the usage of the classifier attribute to differentiate between community and production releases. With this proposal the current CD procedure would be left unchanged and consumers who want to subscribe to the latest releases can continue to do so with no changes. For those consumers who want to limit the thrashing on taking updates they can do so by declaring a weekly classifier. The only change to the current release process would to be to release the most recent tag with the additional classifier attached. It could be weekly or monthly.

<!-- Current usage, unchanged! -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.5</version>
</dependency>

<!-- Using the `weekly` classifier -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>weekly</classifier>
    <version>2.7.1</version>
</dependency>

This could also be inverted to restore the delayed release process to the majority of consumers and allow opting in to the daily/continuous releases.

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>daily</classifier>
    <version>2.7.5</version>
</dependency>

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.1</version>
</dependency>

[TimvdLippe]

@mattnelson That is an interesting concept I had not heard of! I am not sure though if it would be valid for our usecase, but definitely worth investigating.

[mockitoguy]

@mattnelson, interesting idea! At this point I want avoid using classifiers because they would complicate things. If you get this working for some project or there is a reference project that uses this method, we would like to know the experiences.

@tinkerware, thanks for feedback!

[TimvdLippe]

@szczepiq I think we can close this issue as well and direct any further discussion to the shipkit repository?

[TimvdLippe]

Closing this per rationale in #618 and our implementation available at https://github.com/mockito/shipkit