feat(tabby): Added --chat-device arg for serve subcommand #1685
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Brief
A implementation of issue #1659
Details
The very simple implementation of the --chat-device flag to override device to use for chat model. This implementation, however, rejects arguments if --chat-device specified without the --chat-model flag. The issue suggests giving a warning instead, but I found out a good
clap
macro to achieve the implemented behaviour. I decided to not implement a logging warning message and give the situation handling to theclap
crate to keep the code clean.Testing
Tested with rocm + cpu devices. I cannot test other devices myself.
closes #1659