Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Ability to specify resource requirements on Actor methods #39910

Open
jfaust opened this issue Sep 27, 2023 · 0 comments
Open

[Core] Ability to specify resource requirements on Actor methods #39910

jfaust opened this issue Sep 27, 2023 · 0 comments
Labels
core Issues that should be addressed in Ray Core core-api enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks

Comments

@jfaust
Copy link

jfaust commented Sep 27, 2023

Description

Currently it's possible to specify resource requirements for Actors:

@ray.remote(memory=1)
class Actor:
  def do_work(...):
    ...

It would be useful to also be able to specify resource requirements on the Actor's methods. Something like:

@ray.remote(memory=1)
class Actor:
  @ray.remote(memory=10)
  def do_work(...):
    ...

Currently to work around it I can create a separate task:

@ray.remote(memory=10)
def do_work(actor, ...):
  return ray.get(actor.do_work(...))

This is not super intuitive, and ends up returning retrieving the results in that task in order to return them, which is not ideal.

I was told to tag @rkooo567

Use case

Initially asked here: https://discuss.ray.io/t/specifying-extra-resources-for-functions-tasks-running-inside-an-actor/12265

PyTorch initialization is costly, so we're keeping an Actor running with the models loaded in memory. We can specify resource constraints on the Actor (in our case a custom gpu_memory resource), however the methods on the Actor will consume more resources, and we don't want to preallocate those to the Actor for a host of reasons.

@jfaust jfaust added enhancement Request for new feature and/or capability triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Sep 27, 2023
@rkooo567 rkooo567 added core Issues that should be addressed in Ray Core P1 Issue that should be fixed within a few weeks core-api and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Issues that should be addressed in Ray Core core-api enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks
Projects
None yet
Development

No branches or pull requests

2 participants