feat: add timeout option for bentoml runner config #2890
Conversation
Codecov Report
@@ Coverage Diff @@
## main #2890 +/- ##
==========================================
- Coverage 70.77% 69.88% -0.89%
==========================================
Files 104 121 +17
Lines 9470 9961 +491
==========================================
+ Hits 6702 6961 +259
- Misses 2768 3000 +232
|
|
I have not actually but can you tell me more about this compat layer? |
Oh just that running this on an older version of bentoml. |
jjmachan
force-pushed
the
feat/runner-timeout
branch
from
0bde657
to
94a8665
Compare
| @property | ||
| def runner_timeout(self) -> int: | ||
| "return the configured timeout for this runner." | ||
| runner_cfg = BentoMLContainer.runners_config.get() | ||
| if self._runner.name in runner_cfg: | ||
| return runner_cfg[self._runner.name]["timeout"] | ||
| else: | ||
| return runner_cfg["timeout"] | ||
|
|
There was a problem hiding this comment.
@bojiang This is how we get the timeout for each runner client
There was a problem hiding this comment.
Is there a case that the runner_name exists in the config but didn't provide a timeout ?
There was a problem hiding this comment.
nope, during initialization BentoMLConfiguration will copy over the default config (including timeout, resources, batching etc) to the runner_name so that situation will not happen.
What does this PR address?
Fixes #2888
eg
by default all runner have a timeout of 300 seconds
Before submitting:
guide on how to create a pull request.
make formatandmake lintscript have passed (instructions)?those accordingly? Here are documentation guidelines and tips on writting docs.
Who can help review?
Feel free to tag members/contributors who can help review your PR.