Propagate extra server args to the gunicorn command #291
Conversation
| str(args.num_workers), | ||
| *envs, | ||
| "model_engine_server.inference.forwarding.http_forwarder:app", | ||
| *extra_args, |
There was a problem hiding this comment.
oh wait hmm I think it might be good to just hardcode in the termination grace period seconds as a default here idk
There was a problem hiding this comment.
extra_args feels like it should be a dict to me (e.g. --key1 val1 --key2 val2 from {key1: val1, key2: val2}), is there such thing as extra_kwargs?
this way we can set a default of graceful-timeout and override it if necessary
There was a problem hiding this comment.
Ah, so default it to 600 seconds? Will do! :)
There was a problem hiding this comment.
Oh wait didn't see the second comment when I commented ^that (GitHub is bad at refreshing lol 😅 ). From local testing, parser.parse_known_args() will turn --key1 val1 --key2 val2 into extra_args = ["--key1", "val1", "--key2", "val2"], which seems fine to directly pass? :P
There was a problem hiding this comment.
yeah it's fine to pass; I was hoping for a dict but if we don't get that from argparse then we can just directly pass it
|
|
||
|
|
||
| def start_gunicorn_server(port: int, num_workers: int, debug: bool) -> None: | ||
| def start_gunicorn_server( |
There was a problem hiding this comment.
wait this is the gateway right? I don't think we need to change things for the gateway worker?
There was a problem hiding this comment.
Oh I just searched "gunicorn" and made the change everywhere :P Will undo this! :)
No description provided.