Build documentation and storage-operator inside containers #1846
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Both built and converted packages depends on the same set of tasks (create the destination directory and build the builder image). Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Mimic what was done for packages. Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Since we will add more builder images (for doc, for Go stuff, …), let's add a module dedicated to them. Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Most of the images are built using the directory of the Dockerfile as build context, but some images need a different context (like the image to build the documentation: it needs the `tox.ini` which lives at the root of the repository, while the Dockerfile lays in the `docs` sub-directory). Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
We want to build the documentation inside a container. That way: - peoples can build PDF documentation without having to install a huge LaTeX distribution. - we can simplify the CI workflow (can build the doc from inside the builder image because there is no need for extra dependencies), e.g. removing CI-specific hack in the buildchain code. Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Some classes are really too simple (that's why pylint was complaining about `too-few-public-methods`) so let's replace them by simple functions (less boilerplate + pylint doesn't complain anymore) Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
This will allow to have a better up-to-date definition for the images we build but that are not shipped in the ISO (for now we are using a `touch`ed file, which is perfectible™). Refs: #839 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Until now, we were relying on a `touch`ed file on disk to know if we had built the Docker image. This is perfectible™ and doesn't handle the case where one delete manually the image through Docker. This commit adds a custom `uptodate` predicate that checks if the image is present in the local Docker registry, which is more reliable. Closes: #839 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
If we want to build the docs inside a container, we must chown back the generated files, otherwise they will be owned by root. Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
We can no longer relies on `get_entrypoint_mount` because we will have entrypoint unrelated to packaging. So, instead of using a set of constant for the default configuration we introduce a new helper that generate a default configuration from the entrypoint path (for packaging entrypoints we had new constants because they are referenced in several files). Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
This reverts commit f2b6b4b. We will now use a container to build the documentation, thus we can now build the doc during the `build` step in the CI instead of reusing the doc generated during the `doc` step. Eventually, we will get rid of this `doc` step. Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
This will avoid the installation of deps locally for the devs. Refs: #1830 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Like for documentation, avoid requiring people to install Go (and the deps for the Operator SDK) if they just want to build the ISO (and not actually hack on the code). Will also ensure that everyone use a single version of Go to build, which avoids weird discrepencies (see Go 1.12 vs Go 1.13 behavior w.r.t. the fix for the thrift apache repository…) Refs: #1831 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
The Go build container use Go 1.13, update the go.{mod,sum} accordingly.
Refs: #1831
Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
If we want to build the docs inside a container, we must chown back the generated files, otherwise they will be owned by root. Note that since we are using an Alpine image, we are using `sh` (`bash` is not installed by default), but given the simplicity or our script that's not an issue. Refs: #1831 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
The targets module imports too many things, and we end up with a cyclic
import:
buildchain.builder
-> buildchain.targets
-> buildchain.targets.operator_image
In other words: `builder` import `operator_image` through `targets`, but
`operator_image` also import `builder` (to use the GO_BUILDER`): cycle!!
Closes: #1831
Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
Continuation of the previous commit. Refs: #1831 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
We no longer depends on Go and Operator SDK to build the ISO (because we use a container). Closes: #1831 Signed-off-by: Sylvain Laperche <sylvain.laperche@scality.com>
slaperche-scality
added
topic:ci
Hello slaperche-scality,My role is to assist you with the merge of this Status report is not available. |
Waiting for approvalThe following approvals are needed before I can proceed with the merge:
Peer approvals must include at least 1 approval from the following list:
|
github-actions
bot
added
kind:dependencies
jbertran
approved these changes
Oct 4, 2019
Thanks 🙂 Everything but the You may want to reopen a PR for that part (seems like the weird CI you had disappeared, maybe related to the |
|
/approve |
In the queueThe changeset has received all authorizations and has been added to the The changeset will be merged in:
The following branches will NOT be impacted:
There is no action required on your side. You will be notified here once IMPORTANT Please do not attempt to modify this pull request.
If you need this pull request to be removed from the queue, please contact a The following options are set: approve |
|
I have successfully merged the changeset of this pull request
The following branches have NOT changed:
Please check the status of the associated issue None. Goodbye slaperche-scality. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Component:
buildchain, ci
Context:
Some people just want to build a MetalK8s ISO and be done with it, thus requiring them to install developer dependencies such as Go toolchain and LaTeX distribution is perfectible™
Summary:
Note that we only move build-related task inside a container, things such as live doc rendering and Go codegen/linting still requires you to have dev dependencies (which is acceptable).
Acceptance criteria:
You can build an ISO without Go, Operator SDK or LaTeX installed on your machine.
Closes: #839
Closes: #1830
Closes: #1831