-
Notifications
You must be signed in to change notification settings - Fork 776
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TEST/CI] Add tests covering TensorflowSavedModelArtifact #721
Comments
Hi @parano. I'm a student in the MLH Fellowship program. I'd like to begin working on this issue. Thanks! :) |
It looks like the requirements have updated, and #876 no longer fixes this. I will make the new changes to address the updated issue. :) |
Two questions:
|
I am having some difficulty developing a simple TF 1.14 example. I have only used TF2, so it is a bit challenging to search through old TF1 documentation! 😛 Because the test I am writing involves saving and loading a TF1 model artifact, I have run into this warning: BentoML/bentoml/artifact/tf_savedmodel_artifact.py Lines 119 to 140 in 4cae7a3
But I am having trouble figuring out how to give a test model the @bojiang: I think you wrote the warning in #421. Do you think this will cause some problems writing a TF1 test? Or, do you know of a good workaround? Thanks much! :) |
I am a bit curious about the motivation behind supporting TF1 in BentoML, considering the strong push away from TF1 from Google. For clarity, what were your thoughts when the Part of me is wondering if it is more trouble that it's worth to have to take on the maintenance costs of accommodating for legacy TF1 API features (e.g. models which use |
Hi @joshuacwnewton , It took me a while to retrieve an example using non-keras TensorFlow v1 API: In fact, we did not do extra work to make bentoml support tensorflow v1. |
Great work finding this example @bojiang! It looks like it has all the necessary steps, so thank you. I will see what I can do to adapt the steps to a simpler model for testing purposes. 😄 Interestingly, I notice that the example you linked does not use I think not using BentoML/bentoml/artifact/tf_savedmodel_artifact.py Lines 294 to 298 in 2b6d677
By using |
@bojiang has already implemented tests for V2, and planning to add TF V1 tests |
Currently, there isn't any integration test covering the
TensorflowSavedModelArtifact
:Similar to PR [CI] Include onnx artifact test on travis #726, add a unit test assuming tensorflow pypi package installed, and create a BentoService with TensorflowSavedModelArtifact, test the major functionalities including save, load, run inference
Build a docker container with the saved bundle and test the API server endpoint, see related PR Add integration tests #855 and [CI] Refactor API Server integration tests #871 which have some utility function code that can be re-used for this
Add a script
./travis/run-tensorflow-v2.2-tests.sh
that runspip install tensorflow==2.2
andpytest tests/test_tensorflow_v2.2_model_artifact.py ...
Add another script
./travis/run-tensorflow-v1.14-tests.sh
that runspip install tensorflow==1.14
andpytest tests/test_tensorflow_v1.14_model_artifact.py ...
Add two separate task in travis.yml that launches the two test scripts above https://github.com/bentoml/BentoML/blob/master/.travis.yml
The text was updated successfully, but these errors were encountered: