Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-31180] [Test Infrastructure] Fail early when installing minikube and check whether we can retry #23497

Closed
wants to merge 11 commits into from

Conversation

victor09091
Copy link
Contributor

What is the purpose of the change

Fail early when installing minikube and check whether we can retry.

Brief change log

  • *add a retry loop and fail early if CI didn't manage to install the binaries, *

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)

@flinkbot
Copy link
Collaborator

flinkbot commented Oct 9, 2023

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@@ -54,6 +54,17 @@ function setup_kubernetes_for_linux {
chmod +x minikube && sudo mv minikube /usr/bin/minikube
fi

if ! [ -x "$(command -v minikube)" ]; then
Copy link
Contributor

@afedulov afedulov Oct 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the idea behind using https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube instead of https://storage.googleapis.com/minikube/releases/ ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I misunderstood before, I understood to change another download address

Copy link
Contributor

@afedulov afedulov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description mentions a retry, but I cannot locate any.

Copy link
Contributor Author

@victor09091 victor09091 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @afedulov for the review. Modified to Attempt restart when the download fails

@@ -48,10 +48,19 @@ function setup_kubernetes_for_linux {
sudo rm "$(which minikube)"
fi

# Retry loop download minikube.
local MINIKUBE_DOWNLOAD_RETRY_COUNT=2
for i in $(seq 1 ${MINIKUBE_DOWNLOAD_RETRY_COUNT}); do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentioned it in the other PR already, but just for the sake of transparency: You could use retry_times (see apache/flink:flink-end-to-end-tests/test-scripts/common.sh:883ff) instead of reimplementing the logic.

Would that work?

@victor09091 victor09091 changed the title FLINK-31180 Fail early when installing minikube and check whether we can retry [FLINK-31180] [Test Infrastructure] Fail early when installing minikube and check whether we can retry Oct 20, 2023
@victor09091
Copy link
Contributor Author

Thanks @XComp for the review.

curl -Lo minikube https://storage.googleapis.com/minikube/releases/$MINIKUBE_VERSION/minikube-linux-$arch && \
chmod +x minikube && sudo mv minikube /usr/bin/minikube
download_minikube_url="https://storage.googleapis.com/minikube/releases/$MINIKUBE_VERSION/minikube-linux-$arch"
if ! retry_times ${MINIKUBE_START_RETRIES} ${MINIKUBE_START_BACKOFF} "curl -Lo minikube ${download_minikube_url}"; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have you tried the loop? Looks like curl doesn't return an error code in case of a failure. 🤔 hint: --fail might be what we want according to man curl

@@ -48,10 +48,15 @@ function setup_kubernetes_for_linux {
sudo rm "$(which minikube)"
fi

# Retry loop download minikube.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment doesn't add any additional value. The code below makes it quite clear that retries are done. I guess we can remove the comment again.

@XComp
Copy link
Contributor

XComp commented Oct 20, 2023

Thanks for addressing my comment. It looks like the retry mechanism doesn't work properly, yet. Please see my comments above.

@victor09091
Copy link
Contributor Author

Thanks @XComp for the review. Thank you very much for your suggestion, The current PR code has been submitted on PR #23528, merged together.

@XComp
Copy link
Contributor

XComp commented Oct 23, 2023

Ok, sounds good to me. I'm gonna close this PR in favor of FLINK-32107 and PR #23528

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants