add support for rayserve in apiserver

[scarlet25151]

Why are these changes needed?

Same with ray job, we need to add support for CRUD of ray service in apiserver.

Related issue number

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[scarlet25151]

For now, test on local environment has success and ready for review.

[brucez-anyscale]

thanks. lgtm. Please make sure the tests pass and let @Jeffwan approve

[Jeffwan]

what's the resource field for?

[scarlet25151]

this part here is aligned with the apis of services

kuberay/ray-operator/apis/ray/v1alpha1/rayservice_types.go

Lines 56 to 63 in fdee883

	type RayActorOptionSpec struct {
	RuntimeEnv string `json:"runtimeEnv,omitempty"`
	NumCpus *float64 `json:"numCpus,omitempty"`
	NumGpus *float64 `json:"numGpus,omitempty"`
	Memory *int32 `json:"memory,omitempty"`
	ObjectStoreMemory *int32 `json:"objectStoreMemory,omitempty"`
	Resources string `json:"resources,omitempty"`
	AcceleratorType string `json:"acceleratorType,omitempty"`

the resource definition here is custom define resource in the ray serve schema

[Jeffwan]

Minor: This is actually the custom resource? I feel ray serve's definition is not that clear. As a user, I may be confused on cpu,memory,gpu and resources here. I can not figure out what to put. Do you have the same feeling? If we can use different name to clarify that, that would be great.

[scarlet25151]

yes, I've replace the name with cpu_per_actor ... and add more comments for this field.

[Jeffwan]

@scarlet25151

1. Could you give a comparison between RayServe yaml and HTTP POST payload and that's easier to compare.
2. Let's also make sure docs are updated.
3. I do see few TODO items there, are they blockers for RayServe endpoint? If so, let's add the status summary in the PR description

[scarlet25151]

@Jeffwan

1. Could you give a comparison between RayServe yaml and HTTP POST payload and that's easier to compare.

yes, I will add it to the top of the conversation.

2. Let's also make sure docs are updated.

this part is not updated yet, I will update in the third commit.

3. I do see few TODO items there, are they blockers for RayServe endpoint? If so, let's add the status summary in
   the PR description

here is one things I would like to discuss, should we use the raycluster's ingress or rayservice ingress here?

kuberay/ray-operator/controllers/ray/rayservice_controller.go

Lines 601 to 603 in fdee883

	// TODO: When start Ingress in RayService, we can disable the Ingress from RayCluster.
	func (r *RayServiceReconciler) reconcileIngress(ctx context.Context, rayServiceInstance *rayv1alpha1.RayService, rayClusterInstance *rayv1alpha1.RayCluster) error {
	if rayClusterInstance.Spec.HeadGroupSpec.EnableIngress == nil || !*rayClusterInstance.Spec.HeadGroupSpec.EnableIngress {

[scarlet25151]

The json body of the request would be like:

{
  "name": "chenyu-test",
  "user": "chenyu.jiang",
  "serveDeploymentGraphSpec": {
      "importPath": "fruit.deployment_graph",
      "runtimeEnv": "working_dir: \"https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip\"",
      "serveConfigs": [{
        "deploymentName": "MangoStand",
        "replicas": 1,
        "userConfig": "price: 3",
        "actorOptions": {
          "cpus": 0.1
        }
      },
      {
        "deploymentName": "OrangeStand",
        "replicas": 1,
        "userConfig": "price: 2",
        "actorOptions": {
          "cpus": 0.1
        }
      },
      {
        "deploymentName": "PearStand",
        "replicas": 1,
        "userConfig": "price: 1",
        "actorOptions": {
          "cpus": 0.1
        }
      },
      {
        "deploymentName": "FruitMarket",
        "replicas": 1,
        "actorOptions": {
          "cpus": 0.1
        }
      },{
        "deploymentName": "DAGDriver",
        "replicas": 1,
        "routePrefix": "/",
        "actorOptions": {
          "cpus": 0.1
        }
      }]
  },
  "clusterSpec": {
  // ...
  }
}

which is compatible to the example of the yaml:

kuberay/ray-operator/config/samples/ray_v1alpha1_rayservice.yaml

Lines 11 to 43 in 40ea21c

	deploymentUnhealthySecondThreshold: 300 # Config for the health check threshold for deployments. Default value is 60.
	serveConfig:
	importPath: fruit.deployment_graph
	runtimeEnv: |
	working_dir: "https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip"
	deployments:
	- name: MangoStand
	numReplicas: 1
	userConfig: |
	price: 3
	rayActorOptions:
	numCpus: 0.1
	- name: OrangeStand
	numReplicas: 1
	userConfig: |
	price: 2
	rayActorOptions:
	numCpus: 0.1
	- name: PearStand
	numReplicas: 1
	userConfig: |
	price: 1
	rayActorOptions:
	numCpus: 0.1
	- name: FruitMarket
	numReplicas: 1
	rayActorOptions:
	numCpus: 0.1
	- name: DAGDriver
	numReplicas: 1
	routePrefix: "/"
	rayActorOptions:
	numCpus: 0.1

[Jeffwan]

here is one things I would like to discuss, should we use the raycluster's ingress or rayservice ingress here?

Do you think rayserve should leverage raycluster's ingress to accept external traffic?

[scarlet25151]

here is one things I would like to discuss, should we use the raycluster's ingress or rayservice ingress here?

Do you think rayserve should leverage raycluster's ingress to accept external traffic?

I see, here is an outdate comment, for now I've implement in the way to poulate response with active rayservice cluster's ingress and endpoints .