[RayJob][Status][7/n] Define JobDeploymentStatusNew explicitly

[kevin85421]

Why are these changes needed?

JobDeploymentStatusNew indicates that this is a new RayJob CR, similar to NewState in the spark-on-k8s-operator.

Related issue number

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

[architkulkarni]

Why do we need this? Maybe you can add a brief PR description.

[architkulkarni]

I guess the goal is to move all logic inside the switch statement. That sounds fine to me

[kevin85421]

I guess the goal is to move all logic inside the switch statement. That sounds fine to me

Sorry. I have already updated the PR description. I plan to move most logic into a switch statement like syncSparkApplication in the Spark operator to build a well-defined state machine.

[kevin85421]

CI failure is about RayService and has no relationship with RayJob.

[Irvingwangjr]

I guess the goal is to move all logic inside the switch statement. That sounds fine to me

Sorry. I have already updated the PR description. I plan to move most logic into a switch statement like syncSparkApplication in the Spark operator to build a well-defined state machine.

Just out of curiosity, why do we try to use state machine here? In kubernetes, we need to deal with cache stale problem, which means the state itself might be stale and not reliable. What's more, workerQueue will squash some of the events and kuberay controller will only have one single reconcile. Those situations will make controller logic more complicated

[kevin85421]

@Irvingwangjr Thank you for the question! Yes, there is a delay between the informer cache and the Kubernetes API server. I've built a state machine to more clearly define the state transitions. However, this does not mean that the goal state of the RayJob spec is stateful. In addition, the state transition is not determined by the resource events. The Spark operator is a mature K8s operator that supports batch job patterns. It also defines a state machine. I refer a lot from the project. You may be interested in it.