-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First Serializable implementation to support remoting of h1.Model over the wire #19
Conversation
.. to support remoting of Models over the wire. The main problem occurs when we pre-instantiate Keras models within h1.Models, and the latter has to be serialized for parallel execution. The contained Keras models do not serialize properly, and exceptions are thrown at the remote side as a result.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
words = full_class_name.split(".") | ||
module_name = ".".join(words[:-1]) | ||
class_name = words[-1:][0] | ||
the_class = getattr(sys.modules[module_name], class_name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is safer to use importlib
to retrieve the module. we can also move this to util
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed re Utils.
On hold. See milestone info. |
This was being done to integrate training using Ray. This code is out of sync and a separate PR should be created with the required changes. |
.. we still have to make sure the offending (Keras) models are deleted prior to serialization, so that they don't cause exception on the other side.