[serve] Set "memory" to None in ray_actor_options by default

[shrekris-anyscale]

Why are these changes needed?

By default, Serve sets "memory" in ray_actor_options to 0. However, this is simply ignored in the resources_from_resource_arguments() function. Additionally, setting memory to 0 is explicitly disallowed in the to_memory_units() function.

This change sets Serve's default value for "memory" to None, so it uses Ray's default "memory" value.

Related issue number

Closes #23616.

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • New unit tests are added to test_config.py.

[simon-mo]

alternatively, try unset the field if not specified? This should work with any future Ray default values.

[edoakes]

@suquark @shrekris-anyscale I'm not quite understanding the context here. Does this mean that setting memory to 0 in the Ray API isn't supported? Is this a bug in the API? Setting it to 1 seems like a hack to me.

[suquark]

@edoakes yes, Ray "intentionally" does not support 0 memory (but you can set memory=None), limited here:

ray/python/ray/_private/utils.py

Line 357 in 3e80da7

resources["memory"] = ray_constants.to_memory_units(memory, round_up=True)

. But it could be just due to some tech debts, another solution is to support 0 memory and converts memory=None to memory=0

[shrekris-anyscale]

@edoakes @simon-mo In that case, should we set the default "memory" value to None instead of 0 or 1? @simon-mo unsetting all the fields could also work, but Serve's defaults are sometimes different than Ray (e.g. Serve's num_cpus uses 1 instead of the number of virtual cores).

[edoakes]

@shrekris-anyscale yes I think we should just set it to None in that caes

[shrekris-anyscale]

@edoakes I changed the default to None. Do you happen to know what Ray's default "memory" value is?

[suquark]

actually you should not change the options (this changes options, making it harder to understand and debug), you just skip setting memory, that will make memory None.

[shrekris-anyscale]

What do you mean by changes options? Wouldn't setting memory to None and not setting memory at all have identical effects? These ray_actor_options ultimately get passed into an actor's options call, which also sets memory to None by default.

[suquark]

Later PR deprecates this behavior (also see #api-changes): #23127. But it does not matter too much.

[suquark]

also I am thinking if we should be explicit about default values? when users want to set memory to 0, you just set 0 instead of default value; when it is not set, you are using default value, which may not be 0.

I strongly feel it is a bad design if the default value is bound with 0, and you cannot use 0 directly.

[shrekris-anyscale]

I strongly feel it is a bad design if the default value is bound with 0, and you cannot use 0 directly.

This change raises an exception if the user passes in 0. Assuming Ray doesn't use 0 by default, the memory won't be 0 under any case.