Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vagrant: Improvements to provisioning #10660

Merged
merged 4 commits into from Mar 24, 2020

Conversation

pchaigno
Copy link
Member

This pull request makes a few adjustments to provisioning, mostly to speed it up and ease restarts after interruptions:

  1. Prevent partial downloaded files from ending up in the cache.
  2. Cache cfssl binaries.
  3. Allow restarting an interrupted installation.
  4. Allow skipping make clean for Cilium.

Details in commit messages.

@pchaigno pchaigno added pending-review area/misc Impacts miscellaneous areas of the code not otherwise owned by another area. release-note/misc This PR makes changes that have no direct user impact. labels Mar 23, 2020
@pchaigno pchaigno requested review from a team as code owners March 23, 2020 07:24
@maintainer-s-little-helper maintainer-s-little-helper bot added this to In progress in 1.8.0 Mar 23, 2020
common/build.sh Show resolved Hide resolved
@@ -8,7 +8,9 @@ popd > /dev/null

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewers: It looks like this file is actually only used for the dev. VM provisioning. Should it be moved to contrib/vagrant/scripts/?

@pchaigno pchaigno force-pushed the pr/pchaigno/vagrant-provisioning-improvements branch from eceb96d to 0319903 Compare March 23, 2020 07:31
@coveralls
Copy link

coveralls commented Mar 23, 2020

Coverage Status

Coverage increased (+0.01%) to 45.599% when pulling cb109b3 on pr/pchaigno/vagrant-provisioning-improvements into 0f49fc6 on master.

Vagrantfile Outdated Show resolved Hide resolved
@pchaigno pchaigno force-pushed the pr/pchaigno/vagrant-provisioning-improvements branch from 0319903 to f9d0cf7 Compare March 23, 2020 12:52
Copy link
Member

@qmonnet qmonnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit on INSTALL variable doc, but I'm fine either way and PR looks good to merge.

Documentation/contributing/development/dev_setup.rst Outdated Show resolved Hide resolved
During VM provisioning, we cache some downloaded files outside the VM,
to speed up subsequent provisionings. If a cached file is present we
consider that it's safe to reuse. That is however not the case when the
download was interrupted (e.g., because of a network error or because VM
provisioning was interrupted). Ideally we would check the signature of
downloaded files, but the associated projects have different ways of
exposing the signatures.

This commit changes download_to() to download files in /tmp and move
them to the cache only once the download is succesfull. This behavior
requires the use of 'set -e'. The cache may still end with partial files
if provisioning is interrupted while the file is moved out of /tmp. That
operation is however much faster than the download and an interruption
therefore less likely.

Signed-off-by: Paul Chaignon <paul@cilium.io>
Those binaries are a few MB large and caching them cuts a few seconds
from the total provisioning time.

Signed-off-by: Paul Chaignon <paul@cilium.io>
Currently, when an installation is interrupted during the first
provisioning of the dev. VM, the VM must be destroyed and recreated.
Otherwise, because the VM is already up, start.sh assumes that e.g.
Kubernetes is already installed.

This new env. variable allows us to force a new installation. It is
still only useful to restart interrupted installations as it will fail
to re-install a successful installation (some files are curently locked
while used by Kubernetes).

Signed-off-by: Paul Chaignon <paul@cilium.io>
When provisioning the dev. VM, Cilium is currently build from scratch
every time. This is often unnecessary.

This commit removes the 'make clean' operation before building Cilium. It
is still possible to force its execution by using the new env. variable
MAKECLEAN. Avoiding the 'make clean' saves about 3m2s on a 5-minutes
provisioning when Kubernetes & co. are already installed (including boot
time).

Signed-off-by: Paul Chaignon <paul@cilium.io>
@pchaigno pchaigno force-pushed the pr/pchaigno/vagrant-provisioning-improvements branch from f9d0cf7 to cb109b3 Compare March 24, 2020 08:48
@aanm
Copy link
Member

aanm commented Mar 24, 2020

test-me-please

@pchaigno
Copy link
Member Author

test-docs-please

@borkmann borkmann merged commit d512d92 into master Mar 24, 2020
1.8.0 automation moved this from In progress to Merged Mar 24, 2020
@borkmann borkmann deleted the pr/pchaigno/vagrant-provisioning-improvements branch March 24, 2020 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/misc Impacts miscellaneous areas of the code not otherwise owned by another area. release-note/misc This PR makes changes that have no direct user impact.
Projects
No open projects
1.8.0
  
Merged
Development

Successfully merging this pull request may close these issues.

None yet

5 participants