Support git and stable salt-bootstrap on Gentoo. #1500
Conversation
ijansky
force-pushed
the
bootstrap-gentoo
branch
from
d4af0c6 to
48c0c86
Compare
ijansky
force-pushed
the
bootstrap-gentoo
branch
from
48c0c86 to
7ae1dfc
Compare
|
Coyld you please also add Gentoo here? Thanks! By the way, awesome work! |
|
Previously, we tried to use the bootstrap to build pre-salted images for Gentoo but weren't successful. I've since tried with this PR's branch and I was able to build images (based on gentoo/stage3:systemd) for:
Thanks for providing this, @ijansky! |
|
Thanks! I am glad to help. @s0undt3ch PR was updated with the CI configuration. It adds quite a few tests to the pipeline. Is that ok or would a subset be more convenient? @myii this is the next step. :) I will raise a PR with a Gentoo support to the https://gitlab.com/saltstack-formulas/infrastructure/salt-image-builder repository after this PR gets merged. You can check out the work in my fork https://gitlab.com/ijansky/salt-image-builder/-/tree/gentoo-support |
@ijansky Nice work, it will be good to compare the two efforts. Here's a rough outline of what I've used:
Note that building |
| # emerge currently won't write to files that aren't there, so we need to ensure their presence | ||
| touch /etc/portage/package.accept_keywords /etc/portage/package.keywords /etc/portage/package.license /etc/portage/package.unmask /etc/portage/package.use | ||
| # shellcheck disable=SC2086 | ||
| emerge ${EMERGE_FLAGS} "${@}"; return $? | ||
| } | ||
|
|
||
| __gentoo_pre_dep() { |
There was a problem hiding this comment.
Several tests failed due to the 20 minutes per build limit. Some time could be saved by using emerge-webrsync instead of emerge --sync, but that may not be sufficient.
There was a problem hiding this comment.
Should we add a emerge-webrsync to the kitchen config file for Gentoo?
At least its up to date when the test container is created....
There was a problem hiding this comment.
I like the idea of adding emerge-webrsync to the provision_command section, It is in the PR now.
Unfortunately it does not solve the problem. I went through the logs of one of the successful runs and it turned out that kitchen-docker runs emerge --sync when creating the container - see https://github.com/test-kitchen/kitchen-docker/blob/24a54be8dece9d095e0f9285c509def1c186f088/lib/kitchen/docker/helpers/dockerfile_helper.rb#L81-L88
I guess that the solution is to replace emerge --sync by emerge-webrsync in kitchen-docker. I will play with it and open a PR against that code to see if it can be replaced or not.
EDIT: PR to kitchen-docker repository - test-kitchen/kitchen-docker#373
There was a problem hiding this comment.
With the changes in test-kitchen/kitchen-docker#373, more Gentoo builds succeed, but some of them still get cancelled. I guess that the only two options are either to skip running Gentoo tests in the pipeline or raise the 20 minutes build limit. What do you think, @s0undt3ch?
There was a problem hiding this comment.
We actually need to wait for the kitchen-docker release or install from git on the Gemfile
There was a problem hiding this comment.
Hmm, let's bump it to 40 minutes then 😢
There was a problem hiding this comment.
I am afraid that 45 minutes is closer to what these builds need.
- GIT and Salt >= 3000 ~ 17 minutes
- GIT and Salt 2019 ~ 40-45 minute
- Stable and Salt >= 3000 ~ 35-40 minutes
- Stable and Salt 2019 ~ 4045 minutes
We could drop 2019, since this one takes the most time - it needs to build Python 3.6.
There was a problem hiding this comment.
Can't drop 2019, it's still supported...
There was a problem hiding this comment.
@s0undt3ch I was told by various SaltStack employees that the support is only until Phase 3 support ends on the following page:
So it appears that 2019.2 is only supported for two more days. Or is there something I'm misssing?
I suppose the same doesn't apply for the salt-bootstrap, which probably needs to continue until the Extended life support ends.
There was a problem hiding this comment.
I made the job timeout configurable with the default set to 20 minutes unless overridden in a per distro map. Gentoo jobs have the timeout set to 45 minutes at the moment.
Good news is that all jobs succeeded and most Gentoo jobs finished in less than 20 minutes with just a few exceptions. When the support for 2019 gets dropped at some point in the future, the timeout could be decreased to around 30 minutes.
Let me know if I should squash all commits into one to keep the history cleaner.
|
Thank You! |
What does this PR do?
This PR fixes and improves Salt boostrap on Gentoo. The current implementation is outdated, does not support the git method, and does not have any tests on Gentoo.
What issues does this PR fix or reference?
It fixes #1499.
Previous Behavior
New Behavior