[EPIC] v2 UX refactoring #1373
Comments
|
|
Hello! If you've got some time I'd love to chat; I'm about to open source a package that I think might be interesting and / or useful. Would this be an appropriate venue? |
|
For context I think it might be useful to integrate here. |
* fix(defaults): set better defaults for inferencing This changeset aim to have better defaults and to properly detect when no inference settings are provided with the model. If not specified, we defaults to mirostat sampling, and offload all the GPU layers (if a GPU is detected). Related to #1373 and #1723 * Adapt tests * Also pre-initialize default seed
Certain engines requires to know during model loading if the embedding feature has to be enabled, however, it is impractical to have to set it to ALL the backends that supports embeddings. There are transformers and sentencentransformers that seamelessly handle both cases, without having this settings to be explicitly enabled. The case sussist only for ggml-based models that needs to enable featuresets during model loading (and thus settings `embedding` is required), however most of the other engines does not require this. This change disables the check done at code side, making easier to use embeddings by not having to specify explicitly `embeddings: true`. Part of: #1373
Certain engines requires to know during model loading if the embedding feature has to be enabled, however, it is impractical to have to set it to ALL the backends that supports embeddings. There are transformers and sentencentransformers that seamelessly handle both cases, without having this settings to be explicitly enabled. The case sussist only for ggml-based models that needs to enable featuresets during model loading (and thus settings `embedding` is required), however most of the other engines does not require this. This change disables the check done at code side, making easier to use embeddings by not having to specify explicitly `embeddings: true`. Part of: #1373
This is a tracker for a small overhaul in term of UX of LocalAI. This includes logging, debugging, error messages and user interaction including model galleries. There are at least enough issues to treat this as an epic where we can collect feedback and try to find a strategy on how to improve the user experience with LocalAI, this spans from logging to have detailed error messages, up to how an user configures LocalAI models.
Help is welcome and this is a call for contributors and to help into getting LocalAI better. There are at least few issues that fall into scope:
curlreturn early if model isn't local, with better message #1076The text was updated successfully, but these errors were encountered: