Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e: add parallel VM installation #608

Merged
merged 14 commits into from
Jan 19, 2023
Merged

Conversation

ldevulder
Copy link
Contributor

@ldevulder ldevulder commented Jan 6, 2023

Should fix #148.

Verification run:

NOTE: I also tested with 20 nodes but it fails all the time and also with 10 nodes on K3s. But as the CI version (with 5 nodes) works well I think that we can merge this PR and I will check why more nodes are failing and fix this in another PR.

@ldevulder ldevulder force-pushed the e2e-add-parallel-vm-installation branch 3 times, most recently from d57174e to 33ad3a0 Compare January 6, 2023 14:37
@ldevulder ldevulder marked this pull request as draft January 6, 2023 14:48
@ldevulder ldevulder force-pushed the e2e-add-parallel-vm-installation branch 2 times, most recently from 27eb674 to 1ca5f49 Compare January 9, 2023 15:24
@ldevulder ldevulder self-assigned this Jan 10, 2023
@ldevulder ldevulder force-pushed the e2e-add-parallel-vm-installation branch 21 times, most recently from 28f20b4 to 9d3e551 Compare January 16, 2023 18:31
@ldevulder ldevulder force-pushed the e2e-add-parallel-vm-installation branch 8 times, most recently from 9bd1dd2 to 3106b0a Compare January 18, 2023 17:06
As the docs have been moved in his own repository.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
This is needed to be able to install multiple nodes in parallel.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
Allows node bootstrapping to be done in parallel for worker pool.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
Store each VM disk image in a dedicated storage pool to avoid locking
during parallel bootstrapping.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
Conditions are not exactly the same when multiple nodes are in Waiting
state.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
Signed-off-by: Loic Devulder <ldevulder@suse.com>
Signed-off-by: Loic Devulder <ldevulder@suse.com>
Needed because the parallel bootstrapping imposes more operations in
parallel and so the cluster can take more time to deployed.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
This is to avoid sporadic context issues with goroutine. Better to check
for the same string on all nodes instead of the hostname.
And in fact the hostname set on the node is not very important at this
point.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
Change to the correct variable name.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
Sounds logical, but was not done on RKE2.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
As the bootstrapping logs are now exported, no needs to have them in the
regular CI process. This will reduce the logs on GitHub and also make
them more readable.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
As this is not needed, only worker role should be set.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
@ldevulder ldevulder force-pushed the e2e-add-parallel-vm-installation branch from 3106b0a to 238964d Compare January 19, 2023 08:53
@ldevulder ldevulder marked this pull request as ready for review January 19, 2023 10:05
.github/workflows/master-e2e.yaml Outdated Show resolved Hide resolved
.github/workflows/master-e2e.yaml Show resolved Hide resolved
Copy link
Contributor

@davidcassany davidcassany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍

As it is broken because the `elemental-node-image` used is too old and
not compatible with newer Elemental version.

This will be re-worked and re-added in a near future.

Signed-off-by: Loic Devulder <ldevulder@suse.com>
@ldevulder ldevulder force-pushed the e2e-add-parallel-vm-installation branch from 238964d to 0f0b11f Compare January 19, 2023 10:56
@ldevulder ldevulder merged commit d88e542 into main Jan 19, 2023
@ldevulder ldevulder deleted the e2e-add-parallel-vm-installation branch January 19, 2023 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

e2e: try to parallelize VM installation in Ginkgo e2 tests
3 participants