Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

istio-pilot may skip event handlers when deployed with the Charmed Kubeflow bundle #113

Closed
DnPlas opened this issue Sep 1, 2022 · 3 comments
Closed

istio-pilot may skip event handlers when deployed with the Charmed Kubeflow bundle #113

DnPlas opened this issue Sep 1, 2022 · 3 comments
Labels
bug Something isn't working Kubeflow 1.7
Projects
MLOps Solution Issues

Comments

@DnPlas
Copy link
Contributor

DnPlas commented Sep 1, 2022 •
edited

Behaviour

When deployed in a large bundle with multiple charms related to it (like the Charmed Kubeflow bundle), istio-pilot may skip the method that sends gateway-info to remote units and/or the handler for the default gateway, which causes the following:

  1. If the handle_default_gateway is not called, the default-gateway never gets created.
  2. If the handle_gateway_info_relation is not called, related remove units will never get the gateway info, which prevents them to complete their start up.

Note that even though this is wrong behaviour, istio-pilot will not fail.

Workarounds

For 1): juju run --unit istio-pilot/0 -- "export JUJU_DISPATCH_PATH=hooks/config-changed; ./dispatch"

For 2) This behaviour should be corrected by this change
@DnPlas DnPlas mentioned this issue Sep 1, 2022
Merged
@agathanatasha agathanatasha added this to Needs Triage in MLOps Solution Issues Sep 2, 2022
@agathanatasha
Copy link
Contributor

Related juju bug

@agathanatasha agathanatasha added the bug Something isn't working label Sep 2, 2022
@agathanatasha agathanatasha moved this from Needs Triage to Labeled in MLOps Solution Issues Sep 2, 2022
@DnPlas DnPlas changed the title istio-pilot skips event handlers when deployed with the Charmed Kubeflow bundle istio-pilot may skip event handlers when deployed with the Charmed Kubeflow bundle Sep 5, 2022
@DnPlas DnPlas mentioned this issue Sep 6, 2022
Closed
@DnPlas DnPlas mentioned this issue Sep 22, 2022
Closed
@software-artisan software-artisan mentioned this issue Sep 27, 2022
Closed
@misohu misohu mentioned this issue Oct 24, 2022
Closed
@i-chvets
Copy link
Contributor

This issue is still seen. Need to address it in Istio Event Handling epic.

@DnPlas DnPlas closed this as completed Mar 13, 2023
MLOps Solution Issues automation moved this from Labeled to Done Mar 13, 2023
@DnPlas DnPlas reopened this Mar 13, 2023
MLOps Solution Issues automation moved this from Done to Needs Triage Mar 13, 2023
@DnPlas DnPlas moved this from Needs Triage to In Jira in MLOps Solution Issues Mar 13, 2023
@ca-scribner ca-scribner mentioned this issue Apr 24, 2023
Merged
@ca-scribner
Copy link
Contributor

Closed by #261

MLOps Solution Issues automation moved this from In Jira to Done May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working Kubeflow 1.7
Projects
MLOps Solution Issues
Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@agathanatasha @DnPlas @ca-scribner @i-chvets