Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-16838] Change base image name and apply versioning #78

Closed
wants to merge 5 commits into from

Conversation

tzulitai
Copy link
Contributor

@tzulitai tzulitai commented Mar 28, 2020

This PR has the following end-goal in mind:

  • Examples / E2E tests / quickstart archetype built or run from snapshot versions (i.e. from master or release-2.0 branches) should always run against a locally-built StateFun base image, built from the source of said snapshot version.
  • Released examples / E2E tests / quickstart archetypes, should not require users to locally build a StateFun image, assuming that we will have official images published to Docker Hub.

This PR does a few things to accomplish that:

  1. Let the image build script tools/docker/build-distribution.sh build images tagged with the current source version. i.e.,
    2.1-SNAPSHOT when built from the snapshot master branch, 2.0-SNAPSHOT when built from the snapshot release-2.0 branch, or
    simply 2.0.0 if built from a officially released source distribution

  2. Update Dockerfiles of all examples / E2E tests / quickstart archetype to use a specific version tag. This has the effect that, if those were built from a released distribution, the image may be pulled directly from Docker Hub (when we publish the images after the release). Otherwise, if those were built from a snapshot version, they would use a locally-built snapshot image.

On the side, this PR also uses the opportunity to rename the image name from statefun to flink-statefun, with the following reasoning:

  • It would be more consistent with the naming convention of other distributions, like the Python SDK
  • Lets the image name to show association with Flink

Verifying

I verified the changes by:

  • Running all the examples that used images built from StateFun base image, and all ran without problems
  • Run all end-to-end tests, using mvn clean verify -Prun-e2e-tests
  • Created new project from quickstart archetype - the generated skeleton project has the correct Dockerfile:
FROM flink-statefun:2.1-SNAPSHOT

RUN mkdir -p /opt/statefun/modules/my-app
COPY target/my-app*jar /opt/statefun/modules/my-app/

We also change the image name from statefun to flink-statefun, to have a
more consistent naming with other distributions like the Python SDK.
…version tag

With this change, the quickstart archetype now always uses a specific
image version.

If the archetype is a released artifact, then the version tag used will
be x.y.z, e.g. 2.0.0. Images in this case should be available in the
Docker Hub registry.

Otherwise, if the archetype is a snapshot distribution built from
development branches, then the version tag used wil be snapshot
versions, e.g. 2.1-SNAPSHOT. In this case, you would have to locally
build the image from the corresponding source version.
@@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

FROM statefun
FROM flink-statefun:2.1-SNAPSHOT
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note:
these will be changed to FROM flink-statefun:2.0-SNAPSHOT when backporting these changes to the snapshot release-2.0 branch.

On release candidates / releases, these will be e.g. FROM flink-statefun:2.0.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated update_branch_version.sh script takes care of updating these strings automatically.

Copy link
Contributor

@igalshilman igalshilman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
This looks good to me!

@tzulitai
Copy link
Contributor Author

Thanks for the fast review Igal.

Merging ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants