Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mockito Continuous Delivery Pipeline 2.0 #911

Closed
mockitoguy opened this issue Jan 29, 2017 · 11 comments
Closed

Mockito Continuous Delivery Pipeline 2.0 #911

mockitoguy opened this issue Jan 29, 2017 · 11 comments
Assignees

Comments

@mockitoguy
Copy link
Member

@mockitoguy mockitoguy commented Jan 29, 2017

Proposed changes on easy-to-grasp diagrams are documented on Mockito blog article.

Current release model

Every code change results in a new version in Central repository (not snapshot, a real version). More information on Continuous Delivery in Mockito.

Given high rate of new versions, the community and current tooling is not comfortable in taking in new Mockito versions at fast pace. More feedback and the discussion in issue #618.

Proposed changes

Release every change but not all releases go to Central. Push to standard repository (JCenter / Maven Central) only substantial releases. Push remaining versions to less prominent, but still public space for early adopters.

Glossary: What are major, minor, patch versions? Explanatory example: Version 2.6.5 has major=2, minor=6, patch=5.

  1. Every merged pull request produces new version that lands in Bintray "mockito/maven" repo.
  2. Some versions we produce are "notable versions", by default those are new major (3.0.0, 4.0.0) and new minor versions (3.1.0, 3.2.0). In contrast to regular new versions, the notable versions are included in well-known central repositories: JCenter and Maven Central.

For more details, see the wiki page paragraph on continuous delivery.

Action list

Optional:

  • - release notes have clean format (mockito/shipkit#3)
  • - notable release (target repo) can be forced by commit message

Implementation details

@mockitoguy mockitoguy changed the title Document and change release cadence Change and document release cadence Jan 29, 2017
@mockitoguy mockitoguy self-assigned this Jan 29, 2017
@mockitoguy mockitoguy changed the title Change and document release cadence Change Mockito release cadence Jan 31, 2017
@TimvdLippe
Copy link
Contributor

@TimvdLippe TimvdLippe commented Jan 31, 2017

Every minor or major version change lands in both

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units). Therefore I would say to not do it automatically, but presumably faster than with patch releases. E.g. minor every week published, patch every month.

Example: “community” repo has 2.8.1, 2.8.2, 2.8.3, at the end of the month, 2.8.3 is promoted to production.

This means that 2.81 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

I'm not completely happy with repository naming "community" and "production"

Maybe trunk and production?

New version is published under following conditions

I would like to add an override here for to be able to push a new version. E.g. not pushing an existing version to production, but publishing a new version without manually editing javadoc to artificially trigger the release mechanism

Overall I agree with the proposal, just some small comments. Glad that we are taking action in this direction 👍

@mockitoguy
Copy link
Member Author

@mockitoguy mockitoguy commented Jan 31, 2017

Great feedback thank you!

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units).

"minor" version (e.g. middle number :) is not automatically bumped by the system. It's us (or the PR submitter) that makes a judgement call to explicitly bump minor version. This decision is based on the gravity of the changes. We don't bump minor version unless it has substantial changes. If it does have substantial changes, it probably should be released to mass audience. At least this is my reasoning. If we don't automatically publish minor versions we can end up with versioning scheme where "community" and "prod" repo have pretty much random versions. If we publish minor versions automatically, "prod" contains announcable, significant versions that add substantial value, e.g. 3.1.0, 3.2.0, 3.3.0 and occasional patch release like 3.3.1. "community" repo would have all changes 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, etc. Thoughts?

This means that 2.8.1 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

The idea is to only include substantial versions in official repo. If the user wants to bisect, he needs configure "community" repo explicitly in pom / build.gradle. In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

Maybe trunk and production?

Let's keep thinking. Some other ideas: all-versions/announcable-versions, all-artifacts/significant-artifacts, early-adopters/slow-adopters.

Given that "minor" version (e.g. middle one) is updated explicitly by the PR submitter, he controls

I would like to add an override here for to be able to push a new version.

Good idea. I will add it to the design.

Really nice feedback. Thanks! We can tweak/iterate the process as we learn more, too.

@TimvdLippe
Copy link
Contributor

@TimvdLippe TimvdLippe commented Jan 31, 2017

We don't bump minor version unless it has substantial changes.

Makes sense, 👍

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

Let's keep thinking.

Latest - Stable?

@arend-von-reinersdorff
Copy link
Contributor

@arend-von-reinersdorff arend-von-reinersdorff commented Jan 31, 2017

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

I can think of two popular projects which sometines skip versions:

  • Maven, missing versions eg 3.3.2, 3.3.4-3.3.8 (and upcoming 3.5.0 will skip 3.4.0)
  • Tomcat, missing versions eg 8.5.10, 8.5.7, 8.0.40
@mockitoguy
Copy link
Member Author

@mockitoguy mockitoguy commented Feb 1, 2017

@arend-von-reinersdorff, thank you for feedback and examples!
@TimvdLippe, currently, I most like following naming scheme for repositories: "all-versions" and "notable-versions".

@mockitoguy mockitoguy changed the title Change Mockito release cadence Mockito Continuous Delivery Pipeline 2.0 Feb 2, 2017
@tinkerware
Copy link

@tinkerware tinkerware commented Feb 8, 2017

This looks good! I agree with the model where minor versions are in release channel (notable-versions) and patch versions are in beta channel (all-versions), modulo hot fixes. If patch versions are pushed to notable-versions at the time they are released, you defeat the purpose of the release channel; your "notable versions" channel would churn multiple times daily again.

Maybe Mockito can push all interleaved patch versions available only on the beta channel to the release channel when the release channel is updated? For example:

Before:

release channel – 2.3.0
beta – 2.3.0, 2.3.1, 2.3.2

After:

release channel – 2.3.0, 2.3.1, 2.3.2, 2.3.3 (hotfix)
beta – 2.3.0, 2.3.1, 2.3.2, 2.3.3

This may go against the principle of least surprise, since someone browsing the version history of release channel will notice that suddenly not one but three new versions became visible. This may be acceptable if bisecting and keeping full version history on the release channel are important for the project. As I said before, I don't have a strong preference for this model, except to say that patch versions should not appear in the release channel at the time of their release unless they are hot fixes.

@mattnelson
Copy link

@mattnelson mattnelson commented Feb 9, 2017

I have an alternative opinion from everything proposed so far. I propose introducing the usage of the classifier attribute to differentiate between community and production releases. With this proposal the current CD procedure would be left unchanged and consumers who want to subscribe to the latest releases can continue to do so with no changes. For those consumers who want to limit the thrashing on taking updates they can do so by declaring a weekly classifier. The only change to the current release process would to be to release the most recent tag with the additional classifier attached. It could be weekly or monthly.

<!-- Current usage, unchanged! -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.5</version>
</dependency>

<!-- Using the `weekly` classifier -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>weekly</classifier>
    <version>2.7.1</version>
</dependency>

This could also be inverted to restore the delayed release process to the majority of consumers and allow opting in to the daily/continuous releases.

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>daily</classifier>
    <version>2.7.5</version>
</dependency>

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.1</version>
</dependency>
@TimvdLippe
Copy link
Contributor

@TimvdLippe TimvdLippe commented Feb 9, 2017

@mattnelson That is an interesting concept I had not heard of! I am not sure though if it would be valid for our usecase, but definitely worth investigating.

@mockitoguy
Copy link
Member Author

@mockitoguy mockitoguy commented Feb 10, 2017

@mattnelson, interesting idea! At this point I want avoid using classifiers because they would complicate things. If you get this working for some project or there is a reference project that uses this method, we would like to know the experiences.

@tinkerware, thanks for feedback!

@TimvdLippe
Copy link
Contributor

@TimvdLippe TimvdLippe commented Oct 3, 2017

@szczepiq I think we can close this issue as well and direct any further discussion to the shipkit repository?

@TimvdLippe
Copy link
Contributor

@TimvdLippe TimvdLippe commented Nov 4, 2017

Closing this per rationale in #618 and our implementation available at https://github.com/mockito/shipkit

@TimvdLippe TimvdLippe closed this Nov 4, 2017
TurpIF added a commit to criteo/android-publisher-sdk that referenced this issue Jul 2, 2020
With Gradle version ordering (https://docs.gradle
.org/current/userguide/single_versions.html#version_ordering), if any
publisher wants to use dynamic version such as 3.7.+ to follow minor
versions, then we should not publish any snapshot releases (see this
feature request: gradle/gradle#8126).

Mockito team found a workaround for this: having two artifacts ID for
the same product: one for development with snapshot releases, one for
customer with no snapshot releases (see
mockito/mockito#911).

During the development, we could have all our product (SDK and adapters)
using the dev releases with dynamic versions. Any snapshot releases is
possible without breaking any changes for any publishers. Once we want
to make a release, only true (customer) releases are used, without any
dynamic versions.

Change-Id: Id7226175546f175557078645ecc09c9049f43ed4
JIRA: EE-1138
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants
You can’t perform that action at this time.