USHIFT-2344: Add layer4 image-blueprints to support creating images with RPMs buillt elsewhere #3012
Conversation
openshift-ci
bot
added
the
approved
|
/retitle USHIFT-2344: Add layer4 image-blueprints to support creating images with RPMs buillt elsewhere |
openshift-ci
bot
changed the title
|
@jogeo: This pull request references USHIFT-2344 which is a valid jira issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/valid-reference
test/bin/build_images.sh
Outdated
| @@ -545,6 +545,10 @@ build_images.sh [-iIsdf] [-l layer-dir | -g group-dir] [-t template] | |||
|
|
|||
| -d Dry run by skipping the composer start commands. | |||
|
|
|||
| -e Extract container images. | |||
There was a problem hiding this comment.
Please, remove this option from help
There was a problem hiding this comment.
We should be able to use the existing rhel92.iso and rhel93.iso images built in layer1 and cached.
Why do we need another ISO?
There was a problem hiding this comment.
QE deploys directly from the ISO in some cases (without a kickstart telling the system to override the ostree image in the ISO with one from a web server). To do that, they need an ISO with a full image with MicroShift in it.
There was a problem hiding this comment.
The only difference of this toml with the other source commits is the missing microshift-test-agent.
We should be able to build it on top of the existing rhel92.iso image.
Did you try naming the file rhel92-microshift-art.toml? It would then using the existing build as a base.
There was a problem hiding this comment.
I would not use -art suffix here, because practically, one can drop anything in place of those RPMs.
What about using -ext or -external in accordance with my comment below.
There was a problem hiding this comment.
I did try using the rhel92 image from from layer1 but since it includes packages that are not provided by RHEL (i.e. microshift-test-agent) it requires building the additional RPM from source. The goal of layer4 is to use packaging from the official build tooling and avoid building locally.
|
|
||
| [[packages]] | ||
| name = "microshift" | ||
| version = "{{ .Env.SOURCE_VERSION }}" |
There was a problem hiding this comment.
I'm not sure we should be reusing the same SOURCE_VERSION setting for something else.
May I suggest using the notion of external rpms for what you're trying to implement.
We could have a special EXTERNAL_SOURCE_VERSION setting that would be valid during the build only if a dedicated directory is populated with RPMs manually.
There was a problem hiding this comment.
The build should fail if such RPMs do not exist.
There was a problem hiding this comment.
We probably didn't pick the best name for that variable to begin with. For CI, it's the version of the source code. The QE team is relying on the fact that copying pre-built RPMs out of brew into a repo in the same location that we would find RPMs if we built from source means they can reuse the existing logic for determining the version from those RPMs.
If we added a new variable, we would have to include the version detection logic, and only selectively fail if the RPMs don't exist and we're supposed to build them. We don't currently link any version detection to any specific layers, which makes the templating and script easier to reason about. I don't think we should change that.
If we renamed the variable PRIMARY_VERSION_UNDER_TEST it would be clearer, but I don't think it's worth making that change just for this PR.
I think reusing the existing variable is reasonable.
| [[packages]] | ||
| name = "systemd-resolved" | ||
| version = "*" | ||
|
|
There was a problem hiding this comment.
Can we add here these packages? They may be useful for troubleshooting. Also, iproute-tc is needed for Stress Testing.
I copy them from blueprint_v0.0.1.toml and added kernel-modules-extra
[[packages]]
name = "iputils"
version = "*"
[[packages]]
name = "bind-utils"
version = "*"
[[packages]]
name = "net-tools"
version = "*"
[[packages]]
name = "iotop"
version = "*"
[[packages]]
name = "strace"
version = "*"
[[packages]]
name = "iproute-tc"
version = "*"
[[packages]]
name = "kernel-modules-extra"
version = "*"
There was a problem hiding this comment.
Are these things we may want to recommend that users install, or are they mostly useful for debugging failures in the test environment?
| [[packages]] | ||
| name = "systemd-resolved" | ||
| version = "*" | ||
|
|
There was a problem hiding this comment.
Can we add here the same extra packages I commented in the rhel92 toml file?
|
|
||
| [[packages]] | ||
| name = "microshift" | ||
| version = "{{ .Env.SOURCE_VERSION }}" |
There was a problem hiding this comment.
We probably didn't pick the best name for that variable to begin with. For CI, it's the version of the source code. The QE team is relying on the fact that copying pre-built RPMs out of brew into a repo in the same location that we would find RPMs if we built from source means they can reuse the existing logic for determining the version from those RPMs.
If we added a new variable, we would have to include the version detection logic, and only selectively fail if the RPMs don't exist and we're supposed to build them. We don't currently link any version detection to any specific layers, which makes the templating and script easier to reason about. I don't think we should change that.
If we renamed the variable PRIMARY_VERSION_UNDER_TEST it would be clearer, but I don't think it's worth making that change just for this PR.
I think reusing the existing variable is reasonable.
| [[packages]] | ||
| name = "systemd-resolved" | ||
| version = "*" | ||
|
|
There was a problem hiding this comment.
Are these things we may want to recommend that users install, or are they mostly useful for debugging failures in the test environment?
There was a problem hiding this comment.
QE deploys directly from the ISO in some cases (without a kickstart telling the system to override the ostree image in the ISO with one from a web server). To do that, they need an ISO with a full image with MicroShift in it.
jogeo
force-pushed
the
art_image_build_w_group1_parent
branch
from
2af6042
to
94e95e4
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dhellmann, jogeo The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@jogeo: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
This PR addresses USHIFT-2344 and provides a layer of blueprints intended to be used with RPMs build by ART.
At this time I do not expect the images built from the layer4 directory to be run in CI. Rather they will be used in the QE lab. Creating a layer4 directory also sets up a storage location for additional blueprint templates with variations the QE team would like to use in their lab.