New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mockito Continuous Delivery Pipeline 2.0 #911

Closed
mockitoguy opened this Issue Jan 29, 2017 · 11 comments

Comments

Projects
None yet
5 participants
@mockitoguy
Member

mockitoguy commented Jan 29, 2017

Proposed changes on easy-to-grasp diagrams are documented on Mockito blog article.

Current release model

Every code change results in a new version in Central repository (not snapshot, a real version). More information on Continuous Delivery in Mockito.

Given high rate of new versions, the community and current tooling is not comfortable in taking in new Mockito versions at fast pace. More feedback and the discussion in issue #618.

Proposed changes

Release every change but not all releases go to Central. Push to standard repository (JCenter / Maven Central) only substantial releases. Push remaining versions to less prominent, but still public space for early adopters.

Glossary: What are major, minor, patch versions? Explanatory example: Version 2.6.5 has major=2, minor=6, patch=5.

  1. Every merged pull request produces new version that lands in Bintray "mockito/maven" repo.
  2. Some versions we produce are "notable versions", by default those are new major (3.0.0, 4.0.0) and new minor versions (3.1.0, 3.2.0). In contrast to regular new versions, the notable versions are included in well-known central repositories: JCenter and Maven Central.

For more details, see the wiki page paragraph on continuous delivery.

Action list

Optional:

  • - release notes have clean format (mockito/shipkit#3)
  • - notable release (target repo) can be forced by commit message

Implementation details

@mockitoguy mockitoguy changed the title from Document and change release cadence to Change and document release cadence Jan 29, 2017

@mockitoguy mockitoguy self-assigned this Jan 29, 2017

@mockitoguy mockitoguy changed the title from Change and document release cadence to Change Mockito release cadence Jan 31, 2017

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Jan 31, 2017

Contributor

Every minor or major version change lands in both

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units). Therefore I would say to not do it automatically, but presumably faster than with patch releases. E.g. minor every week published, patch every month.

Example: “community” repo has 2.8.1, 2.8.2, 2.8.3, at the end of the month, 2.8.3 is promoted to production.

This means that 2.81 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

I'm not completely happy with repository naming "community" and "production"

Maybe trunk and production?

New version is published under following conditions

I would like to add an override here for to be able to push a new version. E.g. not pushing an existing version to production, but publishing a new version without manually editing javadoc to artificially trigger the release mechanism

Overall I agree with the proposal, just some small comments. Glad that we are taking action in this direction 👍

Contributor

TimvdLippe commented Jan 31, 2017

Every minor or major version change lands in both

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units). Therefore I would say to not do it automatically, but presumably faster than with patch releases. E.g. minor every week published, patch every month.

Example: “community” repo has 2.8.1, 2.8.2, 2.8.3, at the end of the month, 2.8.3 is promoted to production.

This means that 2.81 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

I'm not completely happy with repository naming "community" and "production"

Maybe trunk and production?

New version is published under following conditions

I would like to add an override here for to be able to push a new version. E.g. not pushing an existing version to production, but publishing a new version without manually editing javadoc to artificially trigger the release mechanism

Overall I agree with the proposal, just some small comments. Glad that we are taking action in this direction 👍

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Jan 31, 2017

Member

Great feedback thank you!

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units).

"minor" version (e.g. middle number :) is not automatically bumped by the system. It's us (or the PR submitter) that makes a judgement call to explicitly bump minor version. This decision is based on the gravity of the changes. We don't bump minor version unless it has substantial changes. If it does have substantial changes, it probably should be released to mass audience. At least this is my reasoning. If we don't automatically publish minor versions we can end up with versioning scheme where "community" and "prod" repo have pretty much random versions. If we publish minor versions automatically, "prod" contains announcable, significant versions that add substantial value, e.g. 3.1.0, 3.2.0, 3.3.0 and occasional patch release like 3.3.1. "community" repo would have all changes 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, etc. Thoughts?

This means that 2.8.1 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

The idea is to only include substantial versions in official repo. If the user wants to bisect, he needs configure "community" repo explicitly in pom / build.gradle. In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

Maybe trunk and production?

Let's keep thinking. Some other ideas: all-versions/announcable-versions, all-artifacts/significant-artifacts, early-adopters/slow-adopters.

Given that "minor" version (e.g. middle one) is updated explicitly by the PR submitter, he controls

I would like to add an override here for to be able to push a new version.

Good idea. I will add it to the design.

Really nice feedback. Thanks! We can tweak/iterate the process as we learn more, too.

Member

mockitoguy commented Jan 31, 2017

Great feedback thank you!

I do agree for major, but for minor I am not sure. I think we should not immediately release a new minor version when 1 function was added, most notably because we might want to add a range of functions in all separate PRs (since they are separate logical units).

"minor" version (e.g. middle number :) is not automatically bumped by the system. It's us (or the PR submitter) that makes a judgement call to explicitly bump minor version. This decision is based on the gravity of the changes. We don't bump minor version unless it has substantial changes. If it does have substantial changes, it probably should be released to mass audience. At least this is my reasoning. If we don't automatically publish minor versions we can end up with versioning scheme where "community" and "prod" repo have pretty much random versions. If we publish minor versions automatically, "prod" contains announcable, significant versions that add substantial value, e.g. 3.1.0, 3.2.0, 3.3.0 and occasional patch release like 3.3.1. "community" repo would have all changes 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0, etc. Thoughts?

This means that 2.8.1 and 2.8.2 are never available on production? I do think we should push this for 1. consistency in release versions and 2. if someone finds an issue with a version in production, he/she can bisect using the in-between patch releases to see which version contained the error.

The idea is to only include substantial versions in official repo. If the user wants to bisect, he needs configure "community" repo explicitly in pom / build.gradle. In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

Maybe trunk and production?

Let's keep thinking. Some other ideas: all-versions/announcable-versions, all-artifacts/significant-artifacts, early-adopters/slow-adopters.

Given that "minor" version (e.g. middle one) is updated explicitly by the PR submitter, he controls

I would like to add an override here for to be able to push a new version.

Good idea. I will add it to the design.

Really nice feedback. Thanks! We can tweak/iterate the process as we learn more, too.

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Jan 31, 2017

Contributor

We don't bump minor version unless it has substantial changes.

Makes sense, 👍

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

Let's keep thinking.

Latest - Stable?

Contributor

TimvdLippe commented Jan 31, 2017

We don't bump minor version unless it has substantial changes.

Makes sense, 👍

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

Let's keep thinking.

Latest - Stable?

@arend-von-reinersdorff

This comment has been minimized.

Show comment
Hide comment
@arend-von-reinersdorff

arend-von-reinersdorff Jan 31, 2017

Contributor

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

I can think of two popular projects which sometines skip versions:

  • Maven, missing versions eg 3.3.2, 3.3.4-3.3.8 (and upcoming 3.5.0 will skip 3.4.0)
  • Tomcat, missing versions eg 8.5.10, 8.5.7, 8.0.40
Contributor

arend-von-reinersdorff commented Jan 31, 2017

In short, the answer to your question is yes, 2.8.1 and 2.8.2 will never be in "production" repo.

I think for clarity's sake, they should be pushed. It would be weird to browse maven central and see missing versions. Let's see what the others think.

I can think of two popular projects which sometines skip versions:

  • Maven, missing versions eg 3.3.2, 3.3.4-3.3.8 (and upcoming 3.5.0 will skip 3.4.0)
  • Tomcat, missing versions eg 8.5.10, 8.5.7, 8.0.40
@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Feb 1, 2017

Member

@arend-von-reinersdorff, thank you for feedback and examples!
@TimvdLippe, currently, I most like following naming scheme for repositories: "all-versions" and "notable-versions".

Member

mockitoguy commented Feb 1, 2017

@arend-von-reinersdorff, thank you for feedback and examples!
@TimvdLippe, currently, I most like following naming scheme for repositories: "all-versions" and "notable-versions".

@mockitoguy mockitoguy changed the title from Change Mockito release cadence to Mockito Continuous Delivery Pipeline 2.0 Feb 2, 2017

@tinkerware

This comment has been minimized.

Show comment
Hide comment
@tinkerware

tinkerware Feb 8, 2017

This looks good! I agree with the model where minor versions are in release channel (notable-versions) and patch versions are in beta channel (all-versions), modulo hot fixes. If patch versions are pushed to notable-versions at the time they are released, you defeat the purpose of the release channel; your "notable versions" channel would churn multiple times daily again.

Maybe Mockito can push all interleaved patch versions available only on the beta channel to the release channel when the release channel is updated? For example:

Before:

release channel – 2.3.0
beta – 2.3.0, 2.3.1, 2.3.2

After:

release channel – 2.3.0, 2.3.1, 2.3.2, 2.3.3 (hotfix)
beta – 2.3.0, 2.3.1, 2.3.2, 2.3.3

This may go against the principle of least surprise, since someone browsing the version history of release channel will notice that suddenly not one but three new versions became visible. This may be acceptable if bisecting and keeping full version history on the release channel are important for the project. As I said before, I don't have a strong preference for this model, except to say that patch versions should not appear in the release channel at the time of their release unless they are hot fixes.

tinkerware commented Feb 8, 2017

This looks good! I agree with the model where minor versions are in release channel (notable-versions) and patch versions are in beta channel (all-versions), modulo hot fixes. If patch versions are pushed to notable-versions at the time they are released, you defeat the purpose of the release channel; your "notable versions" channel would churn multiple times daily again.

Maybe Mockito can push all interleaved patch versions available only on the beta channel to the release channel when the release channel is updated? For example:

Before:

release channel – 2.3.0
beta – 2.3.0, 2.3.1, 2.3.2

After:

release channel – 2.3.0, 2.3.1, 2.3.2, 2.3.3 (hotfix)
beta – 2.3.0, 2.3.1, 2.3.2, 2.3.3

This may go against the principle of least surprise, since someone browsing the version history of release channel will notice that suddenly not one but three new versions became visible. This may be acceptable if bisecting and keeping full version history on the release channel are important for the project. As I said before, I don't have a strong preference for this model, except to say that patch versions should not appear in the release channel at the time of their release unless they are hot fixes.

@mattnelson

This comment has been minimized.

Show comment
Hide comment
@mattnelson

mattnelson Feb 9, 2017

I have an alternative opinion from everything proposed so far. I propose introducing the usage of the classifier attribute to differentiate between community and production releases. With this proposal the current CD procedure would be left unchanged and consumers who want to subscribe to the latest releases can continue to do so with no changes. For those consumers who want to limit the thrashing on taking updates they can do so by declaring a weekly classifier. The only change to the current release process would to be to release the most recent tag with the additional classifier attached. It could be weekly or monthly.

<!-- Current usage, unchanged! -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.5</version>
</dependency>

<!-- Using the `weekly` classifier -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>weekly</classifier>
    <version>2.7.1</version>
</dependency>

This could also be inverted to restore the delayed release process to the majority of consumers and allow opting in to the daily/continuous releases.

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>daily</classifier>
    <version>2.7.5</version>
</dependency>

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.1</version>
</dependency>

mattnelson commented Feb 9, 2017

I have an alternative opinion from everything proposed so far. I propose introducing the usage of the classifier attribute to differentiate between community and production releases. With this proposal the current CD procedure would be left unchanged and consumers who want to subscribe to the latest releases can continue to do so with no changes. For those consumers who want to limit the thrashing on taking updates they can do so by declaring a weekly classifier. The only change to the current release process would to be to release the most recent tag with the additional classifier attached. It could be weekly or monthly.

<!-- Current usage, unchanged! -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.5</version>
</dependency>

<!-- Using the `weekly` classifier -->
<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>weekly</classifier>
    <version>2.7.1</version>
</dependency>

This could also be inverted to restore the delayed release process to the majority of consumers and allow opting in to the daily/continuous releases.

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <classifier>daily</classifier>
    <version>2.7.5</version>
</dependency>

<dependency>
    <groupId>org.mockito</groupId>
    <artifactId>mockito-core</artifactId>
    <version>2.7.1</version>
</dependency>
@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Feb 9, 2017

Contributor

@mattnelson That is an interesting concept I had not heard of! I am not sure though if it would be valid for our usecase, but definitely worth investigating.

Contributor

TimvdLippe commented Feb 9, 2017

@mattnelson That is an interesting concept I had not heard of! I am not sure though if it would be valid for our usecase, but definitely worth investigating.

@mockitoguy

This comment has been minimized.

Show comment
Hide comment
@mockitoguy

mockitoguy Feb 10, 2017

Member

@mattnelson, interesting idea! At this point I want avoid using classifiers because they would complicate things. If you get this working for some project or there is a reference project that uses this method, we would like to know the experiences.

@tinkerware, thanks for feedback!

Member

mockitoguy commented Feb 10, 2017

@mattnelson, interesting idea! At this point I want avoid using classifiers because they would complicate things. If you get this working for some project or there is a reference project that uses this method, we would like to know the experiences.

@tinkerware, thanks for feedback!

@mockitoguy mockitoguy closed this in #1011 Mar 31, 2017

@TimvdLippe TimvdLippe removed the in progress label Mar 31, 2017

@mockitoguy mockitoguy reopened this Apr 3, 2017

mockitoguy added a commit that referenced this issue Apr 23, 2017

Added 2.7.22 to notable version list
- This is the last version we directly published to maven central
so it is our current 'official' notable version.
- For more information on the progress on Continuous Delivery Pipeline 2.0
see #911
@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Oct 3, 2017

Contributor

@szczepiq I think we can close this issue as well and direct any further discussion to the shipkit repository?

Contributor

TimvdLippe commented Oct 3, 2017

@szczepiq I think we can close this issue as well and direct any further discussion to the shipkit repository?

@TimvdLippe

This comment has been minimized.

Show comment
Hide comment
@TimvdLippe

TimvdLippe Nov 4, 2017

Contributor

Closing this per rationale in #618 and our implementation available at https://github.com/mockito/shipkit

Contributor

TimvdLippe commented Nov 4, 2017

Closing this per rationale in #618 and our implementation available at https://github.com/mockito/shipkit

@TimvdLippe TimvdLippe closed this Nov 4, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment