Skip to content

v1.1.3.3

Compare
Choose a tag to compare
@c0sogi c0sogi released this 29 Jun 07:51
· 16 commits to master since this release
  1. Automatically monitors the underlying Llama.cpp API server process for driving the local LLM model. Introduced a more flexible communication method over the network from the IPC method through Queue and Event in the existing process pool.

  2. Local embedding via Llama.cpp model and huggingface embedding model. For the former, you need to set the embedding=True option when defining LlamaCppModel. For the latter, you need to install pytorch additionally and set a huggingface repository such as intfloat/e5-large-v2 in the value of LOCAL_EMBEDDING_MODEL in the .env file.