[ml-release][no-ci] fix torch tune serve test.

[xwjiang2010]

Signed-off-by: xwjiang2010 xwjiang2010@gmail.com

Why are these changes needed?

The test currently misses wait_for_nodes, resulting in timeout.
https://console.anyscale-staging.com/o/anyscale-internal/projects/prj_qC3ZfndQWYYjx2cz8KWGNUL4/clusters/ses_Q4mHYcB1xS93WatGP7i9jSsy

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[krfricke]

I think the idea behind the golden notebook test is explicitly to test autoscaling. If this doesn't work, we should investigate why this is the case - is it node availability, a problem with the autoscaler, the product, or do we just have to increase the timeout?

cc @matthewdeng for context on golden notebook tests

[krfricke]

Hm, though it does seem that the cluster compute specifies min_nodes:

worker_node_types:
    - name: worker_node
      instance_type: g3.8xlarge
      min_workers: 2
      max_workers: 2
      use_spot: true

[matthewdeng]

I don't recall this explicitly intending to test autoscaling.

1. This was originally added with timeout 1800 in [release] add golden notebook release test for torch/tune/serve #16619.
   1. The wait_for_nodes utility may not have existed at this point in time.
2. Timeout was decreased to 600 in [CI] Reduce cost of golden notebook release test #29609.

[krfricke]

Thanks @matthewdeng for the context.