Retry downloads, respect URL list, validate tar hash #74100
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mtaufen
added
sig/node
k8s-ci-robot
added
release-note-none
RA489
reviewed
Feb 15, 2019
cluster/gce/win1803/common.psm1
Outdated
| } | ||
| } | ||
|
|
||
| # Attempts to downlaod the file from URLs, trying each URL in succession (with retries) |
|
/test ci-kubernetes-e2e-windows-gce |
mtaufen
force-pushed
the
file-download-improvements
branch
2 times, most recently
from
bf6f0af
to
9e1bd0e
Compare
mtaufen
changed the title
mtaufen
force-pushed
the
file-download-improvements
branch
from
9e1bd0e
to
513f6f2
Compare
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
pjh
reviewed
Feb 20, 2019
cluster/gce/win1803/common.psm1
Outdated
| @@ -86,5 +86,69 @@ function Get-InstanceMetadataValue { | |||
| } | |||
| } | |||
|
|
|||
| function Validate-Hash { | |||
There was a problem hiding this comment.
Add a comment that the hash is SHA1, or change the function name to indicate this.
cluster/gce/win1803/common.psm1
Outdated
|
|
||
| # Attempts to download the file from URLs, trying each URL in succession (with retries) | ||
| # until it succeeds. It will loop through the URLs list forever until it has a success. | ||
| # If successful, it will write the file to OutDir, with a |
cluster/gce/win1803/common.psm1
Outdated
| if ($Hash -ne $null) { | ||
| Try { | ||
| Validate-Hash -Hash $Hash -Path $OutFile | ||
| } Catch { |
There was a problem hiding this comment.
There's a couple extra leading spaces here.
| if ($kube_env.ContainsKey('NODE_BINARY_TAR_HASH')) { | ||
| $hash = ${kube_env}['NODE_BINARY_TAR_HASH'] | ||
| } | ||
| MustDownload-File -Hash $hash -OutFile ${tmp_dir}\${filename} -URLs $urls |
There was a problem hiding this comment.
Should we change other Invoke-WebRequest calls in this file to use MustDownload-File?
cluster/gce/win1803/common.psm1
Outdated
| # Attempts to download the file from URLs, trying each URL in succession (with retries) | ||
| # until it succeeds. It will loop through the URLs list forever until it has a success. | ||
| # If successful, it will write the file to OutDir, with a | ||
| # filename matching the base (final) segment of the URL. You can optionall provide a Hash |
cluster/gce/win1803/common.psm1
Outdated
|
|
||
| # TODO(mtaufen): We could probably set this in a more general place. | ||
| # Disable progress bar to increase download speed. | ||
| $ProgressPreference = 'SilentlyContinue' |
There was a problem hiding this comment.
You could move this to Set-PrerequisiteOptions in k8s-node-setup.psm1, or (perhaps better) just move this to be "global" at the top of this file.
There was a problem hiding this comment.
Moved to top of file.
mtaufen
force-pushed
the
file-download-improvements
branch
from
513f6f2
to
7ffe810
Compare
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mtaufen, yujuhong The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
|
Looks like this had a small bug, fix in #74324 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I tested this with the instructions in the README: https://github.com/pjh/kubernetes/blob/gce-windows-cluster/cluster/gce/win1803/README-GCE-Windows-kube-up.md
What type of PR is this?
/kind bug
What this PR does / why we need it:
@yujuhong @pjh