-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Using LocalGPT with external server which hosts ollama (via ngrok) #11
Comments
@0xSynth I moved the discussion to an issue. Please provide some more information: how do you run Ollama on your desktop? Have you tried to send regular curl to the server? |
Thank you for quick response, I run obsydian with local-gpt plugin on my desktop. The server is running ollama, the server is accessible via a ngrok tunel and it works fine with https://github.com/kghandour/Ollama-SwiftUI, also requests are accessible via an API - I've tried requests via curl. I had an issue with CORS, but I've managed to resolve it, however the problem still persists. The response I'm getting is
![]() |
Have you tried to kill Ollama client and run Ollama in serve mode |
Yes I did, without OLLAMA_ORIGINS='*' the requests return with 403, for me the issue is that for some reason after updating the API URL, the request to /api/tags is not being sent, as a result I am not able to specify model different than |
Kapture.2024-01-18.at.23.33.04.mp4 |
I was talking about the server side also, on your screenshots they are visible, however for some reason those are not sent to my server. |
Nothing happen after that? Let's exclude possible causes one by one: could you try to install and use the same scenario with the BMO Chatbot plugin? |
Thank you for your recording, for me this refresh default model does not work for some reason, the request are not being sent to the server, at the same time I am able to run the plugin with |
Ok, in the end I've decided to fork your plugin and change the default model to the one I like to use - works fine, however the issue still remains. |
I would drop support for the Ollama protocol and retain only OpenAI. All engines used for locally deploying LLMs have OpenAI compatibility modes in one form or another, including Ollama. The development effort would be much more useful if focused on adding new features rather than debugging VPNs, firewalls, tunnels, and similar issues. Regarding tunneling, I confirm that SSH tunneling works fine with Ollama (in OpenAI compatibility mode), Exllama (TabbyAPI), Aphrodite, and llama.cpp (in OpenAI mode). |
Originally posted by 0xSynth January 17, 2024
Hi, I can't run ollama mixtral model on my desktop, however I have a server to which I would like to connect (via ngrok), however for some reason I can't, is it a bug? Can you add this?
The text was updated successfully, but these errors were encountered: