Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

xgboost_synthetic in 0.7.0 model doesn't deploy #673

Closed
jlewi opened this issue Nov 1, 2019 · 5 comments
Closed

xgboost_synthetic in 0.7.0 model doesn't deploy #673

jlewi opened this issue Nov 1, 2019 · 5 comments
Labels
area/examples kind/bug lifecycle/stale priority/p0
Projects
0.7.0

Comments

@jlewi
Copy link
Contributor

jlewi commented Nov 1, 2019

With 0.7.0 and the xgboost_synthetic example the model doesn't get deployed. The model is crash looping with

kubectl -n kubeflow-jlewi logs fairing-deployer-mpcjk-598789949b-cqsts
Traceback (most recent call last):
  File "/opt/conda/bin/seldon-core-microservice", line 6, in <module>
    from seldon_core.microservice import main
  File "/opt/conda/lib/python3.6/site-packages/seldon_core/__init__.py", line 2, in <module>
    from .storage import Storage
  File "/opt/conda/lib/python3.6/site-packages/seldon_core/storage.py", line 25, in <module>
    from azure.storage.blob import BlockBlobService
ImportError: cannot import name 'BlockBlobService'
@jlewi jlewi added this to To Do in 0.7.0 via automation Nov 1, 2019
@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the label kind/bug to this issue, with a confidence of 0.97. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

jlewi pushed a commit to jlewi/examples that referenced this issue Nov 2, 2019
Fix the xgboost_synthetic test so it actually runs and produces signal
52b0770 
* The test wasn't actually running because we were passing arguments that
  were unknown to pytest

* Remove the old role.yaml; we don't use it anymore

* Wait for the Job to finish and properly report status; kubeflow/testing#514
  contains the new routine

* The test still isn't passing because of kubeflow#673

* In addition we need to fix the auto deployments kubeflow/testing#444

Related to kubeflow#665
@jlewi jlewi mentioned this issue Nov 2, 2019
Merged
@jlewi jlewi moved this from To Do to Needs Fix in 0.7.0 Nov 2, 2019
@suneeta-mall
Copy link

I ran into this .. the seldon code specifies https://github.com/SeldonIO/seldon-core/blob/master/python/requirements.txt#L19 azure-storage-blob >= 2.0.1 and azure-storage-blob 12.0.0 gets installed but seldon core actually need needs azure-storage-blob==2.1.0 as in 12.XX the BlockBlobService is no longer present

@suneeta-mall
Copy link

see here SeldonIO/seldon-core#1023

k8s-ci-robot pushed a commit that referenced this issue Nov 5, 2019
@jlewi @k8s-ci-robot
Fix the xgboost_synthetic test so it actually runs and produces signal (
e2198ce 
#674)

* Fix the xgboost_synthetic test so it actually runs and produces signal

* The test wasn't actually running because we were passing arguments that
  were unknown to pytest

* Remove the old role.yaml; we don't use it anymore

* Wait for the Job to finish and properly report status; kubeflow/testing#514
  contains the new routine

* The test still isn't passing because of #673

* In addition we need to fix the auto deployments kubeflow/testing#444

Related to #665

* Fix lint.
This was referenced Nov 8, 2019
Closed
Closed
@jlewi
Copy link
Contributor Author

jlewi commented Nov 25, 2019

I have some in progress changes on my branch:
https://github.com/jlewi/examples/tree/fix_xgboost3

I think this fixes the issues but I need to clean it up and submit a PR.

jlewi pushed a commit to jlewi/examples that referenced this issue Nov 25, 2019
Fix issues with the xgboost_synthetic example and deploying the model.
e80ff2b 
* install newer version of fairing
* modify preprocessor to use custom dockerfile
* use newer 0.7 base image.
* Fix endpoint.

Related to:

kubeflow#673 model doesn't deploy its crash looping
Related to kubeflow#655 update example to work with 0.7
@jlewi jlewi mentioned this issue Nov 25, 2019
Merged
k8s-ci-robot pushed a commit that referenced this issue Nov 25, 2019
@jlewi @k8s-ci-robot
Fix issues with the xgboost_synthetic example and deploying the model. (
712c29a 
#682)

* Fix issues with the xgboost_synthetic example and deploying the model.

* install newer version of fairing
* modify preprocessor to use custom dockerfile
* use newer 0.7 base image.
* Fix endpoint.

Related to:

#673 model doesn't deploy its crash looping
Related to #655 update example to work with 0.7

* Add some comments to the notebook.
@stale
Copy link

stale bot commented Feb 23, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot closed this as completed Mar 1, 2020
0.7.0 automation moved this from Needs Fix to Done Mar 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/examples kind/bug lifecycle/stale priority/p0
Projects
No open projects
0.7.0
  
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jlewi @suneeta-mall