Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always set variables for predictive unit and deployment identifiers [Fixes #1449] #1491

Merged
merged 1 commit into from
Mar 3, 2020

Conversation

axsaucedo
Copy link
Contributor

Fixes #1449

Ensures the predictive unit and deployment identifier environment variables are always set by the controller.

@seldondev
Copy link
Collaborator

Mon Mar 2 08:10:53 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1491/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1491 --build=1

@seldondev
Copy link
Collaborator

Mon Mar 2 08:11:00 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1491/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1491 --build=2

@adriangonz
Copy link
Contributor

/approve

@seldondev
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adriangonz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@seldondev
Copy link
Collaborator

Tue Mar 3 20:02:12 UTC 2020
The logs for [pr-build] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1491/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1491 --build=3

@seldondev
Copy link
Collaborator

Tue Mar 3 20:03:44 UTC 2020
The logs for [lint] [4] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1491/4.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1491 --build=4

@seldondev seldondev merged commit a771afb into SeldonIO:master Mar 3, 2020
@nickdgriffin
Copy link

I think this will fix some behaviour I've seen where any new/updated SeldonDeployment always creates/updates the Deployment without these values, then updates it straight after to put them on. This means that you end up having to wait for two pods to become ready, in sequence, as well as wasting cluster resources in the mean time - particularly an issue for large models.

@axsaucedo
Copy link
Contributor Author

Interesting - I wasnt' aware of that bug, so great to hear, and would be good if you could open an issue if you see that the bug / annoyance is not fixed yet @nickdgriffin

@nickdgriffin
Copy link

Looks like we've been able to work around it by applying these values ourselves, but will certainly raise a new issue once this is released if it turns up again. This change might still mean it flaps because of reapplying con.Env = append(con.Env, corev1.EnvVar{Name: machinelearningv1.ENV_PREDICTIVE_UNIT_SERVICE_PORT, Value: strconv.Itoa(int(portNum))}) but I can verify that. It wouldn't be the end of the world to have to pick a port per predictive unit though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Predictive Unit/Deployment Environment Variables not set by operator
4 participants