New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
minimal check for import V3 #3175
minimal check for import V3 #3175
Conversation
# This toolkit is compatible with TensorFlow V2 behavior. | ||
# However, until all toolkits are compatible, we must call `disable_v2_behavior()`. | ||
_tf.disable_v2_behavior() | ||
# in conjunction with minimal package |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! Let's move this function into _tf_utils as it's copy-pasted elsewhere, then there's only one of it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there must be a reason for it to exist everywhere before. If we need to clean it, why now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't you add _lazy_import_tensorflow? It was added in this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, it's a function name to wrap the existing code.
2b0bf17
to
9cb7ae7
Compare
… are built as well. Next attempt at valid setup.cfg file. Got wheels to package correctly. Update. Okay, it all works now locally. Pinned llvm version. one minimal build
address pr concerns address pr concerns address pr concerns further clean add minimal_import in merge disaster add encoding string to this pr remove coreml and resampy restore some file
9cb7ae7
to
89a4170
Compare
for 3 files (drawing, activity, and style), at top-level, I wrap import tensorflow.compat.v1 as _tf
# This toolkit is compatible with TensorFlow V2 behavior.
# However, until all toolkits are compatible, we must call `disable_v2_behavior()`.
_tf.disable_v2_behavior() into, def _lazy_import_tensorflow():
_tf = minimal_package_import_check("tensorflow.compat.v1")
# This toolkit is compatible with TensorFlow V2 behavior.
# However, until all toolkits are compatible, we must call `disable_v2_behavior()`.
_tf.disable_v2_behavior()
return _tf and only for the sound classifier, I wrap _utils.suppress_tensorflow_warnings()
import tensorflow.compat.v1 as _tf
# This toolkit is compatible with TensorFlow V2 behavior.
# However, until all toolkits are compatible, we must call `disable_v2_behavior()`.
_tf.disable_v2_behavior() into def _lazy_import_tensorflow():
# Suppresses verbosity to only errors
_utils.suppress_tensorflow_warnings()
_tf = minimal_package_import_check("tensorflow.compat.v1")
# This toolkit is compatible with TensorFlow V2 behavior.
# However, until all toolkits are compatible, we must call `disable_v2_behavior()`.
_tf.disable_v2_behavior()
return _tf These should not be considered as the duplicate of code because each tool may have its own settings on tf. Leaving it at the beginning of the file will allow people to know what's the tf configuration more clearly. I only add 2 more lines around. I don't believe that these 2 more lines are considered as the liability I shouldn't put here since people may not be able to maintain it because these 2 lines increase the mental burden on them so that they can't understand it. Second, if this is considered as the duplicate of code, we wrap all common imports into on function to remove duplicate of @hoytak how do you think? |
passed internal tests (112066). |
from #3180:
from this PR: The team favors later one is reasonable. Even more labor work and more client code affected, this PR comes with lower risk and liability. |
src/python/turicreate/toolkits/drawing_classifier/drawing_classifier.py
Outdated
Show resolved
Hide resolved
src/python/turicreate/toolkits/image_classifier/image_classifier.py
Outdated
Show resolved
Hide resolved
src/python/turicreate/toolkits/object_detector/object_detector.py
Outdated
Show resolved
Hide resolved
src/python/turicreate/toolkits/sound_classifier/sound_classifier.py
Outdated
Show resolved
Hide resolved
src/python/turicreate/toolkits/style_transfer/style_transfer.py
Outdated
Show resolved
Hide resolved
# This toolkit is compatible with TensorFlow V2 behavior. | ||
# However, until all toolkits are compatible, we must call `disable_v2_behavior()`. | ||
_tf.disable_v2_behavior() | ||
# in conjunction with minimal package |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't you add _lazy_import_tensorflow? It was added in this PR.
@hoytak @TobyRoseman updated in ed670ff, the version of ERROR: <package> has requirement turicreate==6.1.1+minimal, but you'll have turicreate 6.1.1 which is incompatible. |
this 6d80701 will trim the "+minimal" from local version (the downstream version) to prompt the upstream version to user if they use any ML toolkits from the minimal version. |
Good work! Ready to go. |
Branched from #3152.
As it is discussed with @hoytak, @TobyRoseman, and @nickjong in #3156, I provide a simpler version for minimal package lazy import in order to lower the maintenance cost.
update 07/05/2020
This PR aims to provide some instructions to users who install the minimal package but somehow wind up using some ML functionalities.
For example, minimal pkg doesn't have TF installed as its dependency. When a user uses ML toolkit that relies on TF, he/she should see the instruction to install the full package's dependency.
Instead of letting them install each package individually, let them install
turicreate==VERSION
to rely on the full package's requirement list to install all required dependencies.