-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parallelise with ray #113
Parallelise with ray #113
Conversation
…nitiative/pyDVL into parallelise_with_ray
Sorry for the deleted type hints. Upon merging I had a bunch of conflicts and in the attempt to solve them I removed some of your stuff. Will fix it now. |
One change that we could make later if we really want to have an abstract interface for the parallelization would be to switch from decorating the classes with @ray.remote
class ShapleyWorker:
... to for example: class RayWorkerBase:
def __new__(cls, *args, **kwargs):
remote_cls = ray.remote(*args, **kwargs)(cls)
actor = remote_cls.remote()
return actor
class ShapleyWorker(RayWorkerBase):
... Of course, this is just an idea |
I tried your suggestion (the one about RayWorkerBase), but there are some things not working properly. Will keep the decorator for now and leave the abstract interface for the future. Also, I have removed the fake remote method from the worker and coordinator and just ignored the type since it was giving me annoying warnings. |
Looks good to me. Feel free to merge. |
Closes #22