Hi @mudler and LocalAI team, this is a great project! Thank you!
Request:
OpenAI-API-compatible server binary, along the lines of whisper.cpp's whisper-server, perhaps living in examples/server/.
Scope
POST /v1/audio/transcriptions: multipart upload, response_format ∈ json | text | verbose_json. The existing --json output (text + per-word start/end/conf) maps almost 1:1 onto verbose_json.
GET /v1/models returning the loaded GGUF, so stock OpenAI SDKs work unmodified.
References
whisper.cpp server example: https://github.com/ggml-org/whisper.cpp/tree/master/examples/server
llama.cpp llama-server (OpenAI-compatible precedent in the ggml ecosystem): https://github.com/ggml-org/llama.cpp/tree/master/tools/server
Hi @mudler and LocalAI team, this is a great project! Thank you!
Request:
OpenAI-API-compatible server binary, along the lines of whisper.cpp's whisper-server, perhaps living in examples/server/.
Scope
POST /v1/audio/transcriptions: multipart upload, response_format ∈ json | text | verbose_json. The existing --json output (text + per-word start/end/conf) maps almost 1:1 onto verbose_json.
GET /v1/models returning the loaded GGUF, so stock OpenAI SDKs work unmodified.
References
whisper.cpp server example: https://github.com/ggml-org/whisper.cpp/tree/master/examples/server
llama.cpp llama-server (OpenAI-compatible precedent in the ggml ecosystem): https://github.com/ggml-org/llama.cpp/tree/master/tools/server