Please sign in to comment.
cluster: fix plugin pickling bug
The recent plugins kwarg refactored the code to use a list of plugin objects rather than plugin settings from the config. The plugin objects were then being serialized and stored in each node's userdata when adding nodes. This worked, however, if the plugin stored anything that's not serializable after its first run, such as the list of nodes, this leads to errors when later adding nodes in the same process. The loadbalancer in particular is affected by this bug. Fixed this by storing the __init__ kwargs and fully qualified class path used to create the plugin in a __plugin_metadata__ attribute which gets dynamically created via ClusterSetup.__new__. Updated Cluster class to serialize a list of __plugin_metadata__ attributes (stored in Cluster._plugins) rather than the plugin object list itself (stored in Cluster.plugins). Updated Node.get_plugins to iterate through the __plugin_metadata__ attributes and recreate the plugin objects. closes gh-165
- Loading branch information...
Showing with 35 additions and 3 deletions.