Added the option to wait for model & tokenizer in lmql.model.serve

[chrispan68]

Currently the model starts up and runs the server while the model and tokenizer are loading.

The client cannot tell when the model & tokenizer are ready, so they could send requests during this period. However, since models often take a long time to load, if the client sends a request during this period, they get a timeout, and an exception is raised.

A simple solution to this is to add an optional flag that tells the model server not to run until after the model and tokenizer are loaded. That way the client can know not to send requests while the model is loading (since the port is not open)

In some use cases where a python script wants to host a model and query inference, having this optional flag would be really convenient.

Since this is an optional flag, it doesn't change the current behavior at all, but just provides the option to people who want to use it.

[lbeurerkellner]

Thanks, looks good. I would test and then merge this PR.

Just a quick heads up, the core team has decided to relicense LMQL under the Apache 2.0 license, do you agree with merging your changes under this updated license. Let me know.

[chrispan68]

I've tested this locally, and it works well.

I have a python script that hosts the model using POpen and queries the model once the server is running. Without the option, the script errors out on Timeout, after turning the option on, the script properly waits until both the model and tokenizer are running and no longer errors out.

I also agree to merging the changes under the updated license. Once this is approved I can merge it.

[lbeurerkellner]

Thanks, I could just test the fix. Works great. Merging this, thanks a lot.