Update buildchain/CI to build the Storage Operator #1410

slaperche-scality · 2019-07-11T14:59:57Z

Component:

buildchain, CI

Why this is needed:

We need to be able to build and deploy a Kubernetes operator written in Go.

What should be done:

Update the documentation about the new building dependencies (go, operator-sdk 0.9.0, …)
Add new dependencies in the CI build container
Build the container image for the operator
Lint the Go code of the operator (at least enforce go-fmt, maybe linting as well)
Ensure operator-sdk generate k8s and operator-sdk generate openapi don’t result in a diff, which would otherwise mean codegen wasn’t run as part of the commit/changes

Test plan:

Being able to build an ISO that contains the operator container, both locally and on the CI.

The text was updated successfully, but these errors were encountered:

We now need to build the Storage Operator, which we built using the Operator SDK. Consequently, we need to add the Operator SDK to the build image. Since the SDK depends on Go and dep, we also install those. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

That way, we will be able to selectively override one while reusing the other (e.g. keep saving with Skopeo, but build the image using the operator-sdk instead of Docker directly). Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

Until now, the command-name was simply the name of the enum variant in lowercase, but this will no longer be the case with OPERATOR_SDK/operator-sdk. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

Pylint seems to think that `name` is a method instead of a string, which is wrong. See pylint-dev/pylint#2062 Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

Add a new image to be built: the container image for the Storage operator. Since it's very similar to a LocalImage, except for the building part (we use the operator-sdk command instead of Docker directly, because we need to compile the Go code), we introduce a new class (OperatorImage) that inherits from LocalImage and overload the `_do_build` method. Note that we take care of enabling Go module and compiling from the `storage-operator` directory. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

For now, we only check that the code is properly formated using `gofmt`. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

We added new tasks but the schema wasn't updated. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

Until now, lint tasks weren't run alongside "build" task, and thus the discrepancy in the display wasn't a bother. But now, we have this kind of output, which is unsightly: -- lint:python -- lint:shell -- lint:yaml . CHECK CMD gofmt -- lint:go So let's define a custom title for the lint tasks to have something like that instead: -- LINT python -- LINT shell -- LINT yaml . CHECK CMD gofmt -- LINT go Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

For now, we only have a task for Go code. If we ever use `black` for Python, we will add a new subtask here. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

The arguments list was computed twice: once in `lint` and once in `format`. Let's use a constant instead. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

This reduce code duplication between `lint` and `format`. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

Also add a minimum required version for Docker (according to the README of the Operator SDK) Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

We commit the files generated by the `operator-sdk` command, as such we want to make sure that when modifying the "source" file we don't forget to regenerate the scaffolding files. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

For now, I plan to create one task per language, for the sake of consistency with `lint` and `format`. If we have more operator in the future, it may make more sense to have one subtask per operator: we can change when/if it happens. Refs: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>

We were using an operator-sdk straight from the master branch. Now that a proper release was made (0.9.0) we use it and thus should update the already generated files with it. Closes: #1410 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>