Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes 'kfctl apply fails to create k8s resources; tries to deploy to 127.0.0.1' #2813

Merged
merged 2 commits into from Mar 27, 2019

Conversation

kkasravi
Copy link
Contributor

@kkasravi kkasravi commented Mar 26, 2019

fixes #2791


This change is Reviewable

@kkasravi
Copy link
Contributor Author

/assign @jlewi

@kkasravi
Copy link
Contributor Author

/assign @kunmingg

@jlewi
Copy link
Contributor

jlewi commented Mar 26, 2019

Can you remove the retry logic around kfctl apply in the E2E test here:

run_with_retries([kfctl_path, "apply", "-V", "all"], cwd=app_path)

Right now that retry logic is masking problems like this.

If there are legitimate retryable failures we can bad more specific retry logic that only retries when those specific errors occur.

@@ -437,10 +437,16 @@ func (ksApp *ksApp) envSet(envName string, host string) error {
actions.OptionAppRoot: ksApp.ksRoot(),
actions.OptionEnvName: ksApp.KsEnvName,
actions.OptionServer: host,
actions.OptionOverride: true,
Copy link
Contributor

@jlewi jlewi Mar 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this doing? Is this just an idiosyncrancy of ksonnet that they require you set the optionOverride to explicitly tell it that the host associated with the environment should be used as opposed to the one set during init?

})
if err != nil {
return fmt.Errorf("There was a problem setting ksonnet env: %v", err)
}
loadApp, loadErr := app.Load(afero.NewOsFs(), ksApp.KApp.HTTPClient(), ksApp.ksRoot())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. So I'm guessing RunEnvSet is just modifying the config files on disks and we need to explicitly load the app to pick up the changes.

@jlewi
Copy link
Contributor

jlewi commented Mar 26, 2019

Looks good. Only comment is to update the test so that we can verify its fixed and catch errors like this in the future.

Copy link
Contributor Author

@kkasravi kkasravi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, updated to remove retry logic around kfctl apply.

Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @ellis-bigelow, @jlewi, and @kunmingg)


bootstrap/pkg/kfapp/ksonnet/ksonnet.go, line 440 at r1 (raw file):

Previously, jlewi (Jeremy Lewi) wrote…

What's this doing? Is this just an idiosyncrancy of ksonnet that they require you set the optionOverride to explicitly tell it that the host associated with the environment should be used as opposed to the one set during init?

correct, subtle but critical


bootstrap/pkg/kfapp/ksonnet/ksonnet.go, line 445 at r1 (raw file):

Previously, jlewi (Jeremy Lewi) wrote…

I see. So I'm guessing RunEnvSet is just modifying the config files on disks and we need to explicitly load the app to pick up the changes.

also correct.

@jlewi
Copy link
Contributor

jlewi commented Mar 27, 2019

/lgtm
/approve

@jlewi
Copy link
Contributor

jlewi commented Mar 27, 2019

Looks like a tf-serving flake
/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jlewi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@lluunn
Copy link
Contributor

lluunn commented Mar 27, 2019

/retest

@k8s-ci-robot k8s-ci-robot merged commit 55ad7b1 into kubeflow:master Mar 27, 2019
lluunn pushed a commit to lluunn/kubeflow that referenced this pull request Mar 28, 2019
…127.0.0.1' (kubeflow#2813)

* fixes 'kfctl apply fails to create k8s resources; tries to deploy to 127.0.0.1'

* remove run_with_retries for kfctl_go_test kfctl apply use case
@kkasravi kkasravi deleted the kfctl_hostfix branch April 24, 2019 00:19
saffaalvi pushed a commit to StatCan/kubeflow that referenced this pull request Feb 11, 2021
…127.0.0.1' (kubeflow#2813)

* fixes 'kfctl apply fails to create k8s resources; tries to deploy to 127.0.0.1'

* remove run_with_retries for kfctl_go_test kfctl apply use case
saffaalvi pushed a commit to StatCan/kubeflow that referenced this pull request Feb 12, 2021
…127.0.0.1' (kubeflow#2813)

* fixes 'kfctl apply fails to create k8s resources; tries to deploy to 127.0.0.1'

* remove run_with_retries for kfctl_go_test kfctl apply use case
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kfctl apply fails to create k8s resources; tries to deploy to 127.0.0.1
6 participants