[Bug]: I am trying to run ollama/dolphin-phi available on ollama, but /chat/{chat_id}/question throws {"error":"model 'llama2' not found, try pulling it first"}
#2056
Comments
|
🤖 Hello @shubham-dayma! 👋 I'm Dosu, a friendly bot here to help while a human maintainer gets to your issue. I'm great at solving bugs, answering questions, and helping folks like you become contributors. Consider me a friendly helper while we wait for a human touch. I'm currently diving into the issue you've posted and will have a full response for you shortly. Hang tight! 🚀 |
dosubot
bot
added
area: backend
|
|
|
@StanGirard Thanks for looking into this. I already had
Now I am wondering how should I get brain's model in backend/models/settings.py and update the initializing of |
|
🤖 Based on the context provided, it seems like you've already found a solution to the issue by modifying the One way to achieve this could be to modify the class BrainSettings(BaseSettings):
...
model: str = "llama2" # Add this line
class Config:
env_file = ".env"
def get_embeddings():
settings = BrainSettings()
if settings.ollama_api_base_url:
embeddings = OllamaEmbeddings(
base_url=settings.ollama_api_base_url,
model=settings.model # Use the model from settings here
)
else:
embeddings = OpenAIEmbeddings()
return embeddingsThis solution assumes that the Please note that this is a potential solution based on the information provided and the context of the Quivr repository. It might not be the best or the only solution. If you have any further questions or need more help, feel free to ask. For more information, you can refer to the Quivr repository and the FastAPI documentation on settings management. SourcesThis response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. |
embeddings = OllamaEmbeddings(model="XXX") make it your own model, like "qwen:7b"、“llama2” and so on, that will be solved |
|
Exactly!
…On Sun, Mar 17, 2024 at 21:20:51, Wills < ***@***.*** > wrote:
>
>
> @ StanGirard ( https://github.com/StanGirard ) Thanks for looking into
> this. I already had dolphin-phi in ollama. I tried checking ollama logs and
> it ended up that, while initializing langchain.embeddings.ollama.OllamaEmbeddings
> we are not passing param model and by default this is considering model as
> llama2. I changed this code (
> https://github.com/StanGirard/quivr/blob/31ce6c05660db8088da96f7bd6f5a0f5f60ffaf7/backend/llm/rags/quivr_rag.py#L67
> ) as below and it worked.
>
>
>>
>>
>> @ Property ( https://github.com/Property )
>> def embeddings(self):
>> if self.brain_settings.ollama_api_base_url:
>> return OllamaEmbeddings(
>> base_url=self.brain_settings.ollama_api_base_url,
>> *model=self.model.replace('ollama/', "")*
>> ) # pyright: ignore reportPrivateUsage=none
>> else:
>> return OpenAIEmbeddings()
>>
>>
>
>
>
> Now I am wondering how should I get brain's model in backend/ models/ settings.
> py ( http://backend/models/settings.py ) and update the initializing of OllamaEmbeddings
> here (
> https://github.com/StanGirard/quivr/blob/31ce6c05660db8088da96f7bd6f5a0f5f60ffaf7/backend/models/settings.py#L133
> ).
>
>
embeddings = OllamaEmbeddings(model="XXX")
make it your own model, like "qwen:7b"、“llama2” and so on, that will be
solved
—
Reply to this email directly, view it on GitHub (
#2056 (comment) ) , or
unsubscribe (
https://github.com/notifications/unsubscribe-auth/AEVUW3DBZREPV5C7J6UV65LYYZTSHAVCNFSM6AAAAABCFEO5WWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSHEYDCNBQGA
).
You are receiving this because you were mentioned. Message ID: <QuivrHQ/quivr/issues/2056/2002901400
@ github. com>
|
How can I set up a local embedded model if the latest version does not have this file? |
|
come up with the fix #2690 |
What happened?
I am trying to run
ollama/dolphin-phimodel on ollama but /chat/{chat_id}/question throws{"error":"model 'llama2' not found, try pulling it first"}error. I don't want to loadllama2on my ollama due to less resources available on my system.Relevant log output
Twitter / LinkedIn details
No response
The text was updated successfully, but these errors were encountered: