[Docs][SkyServe] Autoscaling doc for SkyServe #2989
Conversation
Co-authored-by: Ziming Mao <ziming.mao@yale.edu>
docs/source/serving/autoscaling.rst
Outdated
|
|
||
| # ... | ||
|
|
||
| The service will scale down all replicas when there is no traffic to the system and will save costs on idle replicas. In this case, the scale up will be faster when the system has no replicas: it will **scale up immediately if any traffic detected**. |
There was a problem hiding this comment.
What does
In this case, the scale up will be faster when the system has no replicas:
mean?
There was a problem hiding this comment.
It means when the service has no replica, user traffic will trigger an immediate scale-up.
There was a problem hiding this comment.
i.e., the upscale delay is ignored?
There was a problem hiding this comment.
Yes. it will immediately change Ntar
There was a problem hiding this comment.
How about rephrasing: When upscaling from zero, the upscale delay will be ignored in order to bring up the service faster.
|
@concretevitamin Those changes look great to me! Added a line on how to adjust scaling delays. |
docs/source/serving/autoscaling.rst
Outdated
|
|
||
| # ... | ||
|
|
||
| The service will scale down all replicas when there is no traffic to the system and will save costs on idle replicas. In this case, the scale up will be faster when the system has no replicas: it will **scale up immediately if any traffic detected**. |
There was a problem hiding this comment.
How about rephrasing: When upscaling from zero, the upscale delay will be ignored in order to bring up the service faster.
Co-authored-by: Zongheng Yang <zongheng.y@gmail.com>
Blocked by #2995 .
Tested (run the relevant ones):
bash format.shpytest tests/test_smoke.pypytest tests/test_smoke.py::test_fill_in_the_namebash tests/backward_comaptibility_tests.sh