Rename ActorConfigured to ActorService

[DNXie]

Due to the side effect of options, now you can see the class names become ActorConfigured (e.g., PolicyConfigured, ReferenceModelConfigured).

Renaming it to ActorService
Now the log becomes

Spawning Service Actor for DatasetActorService
Spawning Service Actor for PolicyService
Spawning Service Actor for TrainerService
Spawning Service Actor for ReplayBufferService
Spawning Service Actor for ComputeAdvantagesService
Spawning Service Actor for RefModelService
Spawning Service Actor for RewardActorService

---

Design Choice 1: Why we have to use a subclass:

We make a subclass of the actor class to store a specific ServiceConfig. This allows multiple different configs to coexist. For example:

service1 = await MyActor.options(num_replicas=2).as_service()
service2 = await MyActor.options(num_replicas=4).as_service()

Both services can run at the same time safely because each call to options() creates a separate subclass with its own config. If we skipped the dynamic subclass and just attached the config to the base class, the second call would overwrite the first config, and service1 would see num_replicas=4 instead of 2.

Design Choice 2: Why we do not make it print the original class name:

We can do it with something like

cfg_cls = type(
    f"{cls.__name__}Configured",
    (cls,),
    {
        "_service_config": cfg,
        "__repr__": lambda self: f"<{cls.__name__}>",
        "__str__": lambda self: f"<{cls.__name__}>",
    },
)

If there were a bug with the Service spawned version instead of the Actor, it would be harder to diagnose.

[joecummings]

Thank you!