Add options for build container rsync optimization #36361
Conversation
k8s-github-robot
added
size/M
ivan4th
force-pushed
the
build-container-rsync-optimizations
branch
2 times, most recently
from
5bbb369
to
a9d8e01
Compare
|
Jenkins unit/integration failed for commit 5bbb3695a7138144c1d50e3ea667f25e004dc8fc. Full PR test history. The magic incantation to run this job again is |
|
Jenkins verification failed for commit 5bbb3695a7138144c1d50e3ea667f25e004dc8fc. Full PR test history. The magic incantation to run this job again is |
|
@k8s-bot gci gce e2e test this |
1 similar comment
|
@k8s-bot gci gce e2e test this |
k8s-github-robot
added
the
do-not-merge
apelisse
removed
the
do-not-merge
|
Sorry for the slow response. I'm inclined to push back on this because it's a set of knobs for the build system that are unlikely to be tested in any continuous integration setting. Unless they have a lot of value being changed by default (which e.g. compression may), I don't think we need more knobs here. cc @jbeda |
|
Does it make sense to copy locally generated files to the build container then? Maybe if knobs aren't acceptable, we may just always skip copying these files? As of compression, I suspect it doesn't add that much overhead and we can enable it by default, wdyt? It helps if you use remote docker that's indeed quite remote. Although if that's problematic I can remove this part & just use ssh compression in such cases, as rsync port is forwarded anyway. |
|
I'm cool with the "compress" option. It is semantically identical and can make sense when you have a remote docker daemon that you are pushing to. Compression should be off, by default as it will slow down the local docker case. It is worth measuring though. Note that right now there is no ssh compression by default as we are doing rsync over the rsync protocol, not via ssh. You'd get ssh compression if you are using a remote daemon with an ssh tunnel, but rsync doesn't know about that. As for copying the generated files back to the build container -- we should probably never do that. I'm okay with changing this so that we only copy generated files FROM the build container and never TO the build container. |
jbeda
added
release-note-none
ivan4th
force-pushed
the
build-container-rsync-optimizations
branch
from
a9d8e01
to
0162b4e
Compare
k8s-ci-robot
added
the
cncf-cla: yes
|
@jbeda updated the PR, now it always ignores generated files when rsyncing to the container. As of compress option, yes, it's off by default. |
|
Jenkins kops AWS e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Jenkins GCE e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Jenkins GCE etcd3 e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Jenkins GKE smoke e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Jenkins GCI GKE smoke e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Jenkins Kubemark GCE e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Jenkins GCI GCE e2e failed for commit 0162b4e42bb4351fb5d5a79e623ceb231c0bbd7c. Full PR test history. The magic incantation to run this job again is |
|
Hmm looks like not copying generated files to the container is not going to work, let me see |
ivan4th
force-pushed
the
build-container-rsync-optimizations
branch
2 times, most recently
from
f7258ab
to
c68f0bd
Compare
|
There was a problem with |
|
Jenkins CRI GCE Node e2e failed for commit c68f0bded6f384c5dd1ac61997693677a22b3762. Full PR test history. The magic incantation to run this job again is |
|
Flake similar to (3) from #35935 (comment) @k8s-bot cri node e2e test this |
KUBE_RSYNC_COMPRESS env var sets rsync compression level. KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER env var disables rsyncing generated files to build containers.
ivan4th
force-pushed
the
build-container-rsync-optimizations
branch
from
c68f0bd
to
2256b27
Compare
|
Jenkins Bazel Build failed for commit 2256b27. Full PR test history. The magic incantation to run this job again is |
|
@k8s-bot bazel test this |
|
ping... |
|
/lgtm Sorry for dropping this. Was waiting for the test issues to be figured out. |
k8s-ci-robot
added
the
lgtm
|
Jenkins CRI GCE e2e failed for commit 2256b27. Full PR test history. The magic incantation to run this job again is |
|
@k8s-bot test this [submit-queue is verifying that this PR is safe to merge] |
|
Automatic merge from submit-queue (batch tested with PRs 38277, 36361, 38452) |
KUBE_RSYNC_COMPRESS env var sets rsync compression level.
KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER env var disables rsyncing
generated files to build containers.
Why KUBE_RSYNC_COMPRESS is needed -- from rsync manual on
--compressoption (implied by non-zero--compress-level):Use case for
KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER: when you sometimes build stuff locally (e.g.make WHAT=cmd/kubectl) and sometimes do it on remote docker (build-tools/run.sh make WHAT=cmd/hyperkube), local builds touch generated files which causes them to be rsynced to the build data container, which may slow down the builds. Still, I'm not sure whether local->remote rsync of generated files is useful (e.g. someone may want to edit generated files for debugging purposes?), so I made not rsyncing these files an option instead of forcing such behavior.This change is