Expose Ollama Service to use it in Chrome Browser Extension

[christophrothmann]

Hey guys,
can you show us, how we can actually use the provided Ollama API's in an chrome extension.
I tried it before, but I get an 403 - Forbidden error. I already looked in the ollama documentation but did not found anything useful.

Hopefully someone can show us how to use it properly. I opened a stackoverflow question before.

Best regards,
Chris

[remy415]

@DevChrisRoth

Is your Ollama instance being served on the same PC? If not, you may need to set the OLLAMA_HOST environment variable to an interface that allows external connections, and the OLLAMA_ORIGINS environment variable to allow cross-origin requests.

OLLAMA_HOST=0.0.0.0 # Listen on all interfaces

I'm not exactly sure how the OLLAMA_ORIGINS environment variable works, try setting it to the machine you're running the chrome extension on.

https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-expose-ollama-on-my-network

[remy415]

@DevChrisRoth

Ollama uses the Gin CORS package to handle origin requests:
https://github.com/gin-contrib/cors?tab=readme-ov-file#canonical-example

The origin will be whatever domain or IP you're hosting the service that will be connecting to the Ollama API. For example, if you have a button on a website that sends data to the Ollama API, and the user will type "https://example.domain.com/ollama" to get to your website, then put "example.domain.com" in your OLLAMA_ORIGINS environment variable.

If you set OLLAMA_ORIGINS=192.168.0.69, your Ollama instance will allow connection requests originating from 192.168.0.69, meaning you're hosting your user interface on that IP address. You can also set it to OLLAMA_ORIGINS=example.domain.com,192.168.0.69, and your Ollama instance will allow connection requests originating from both example.domain.com and the IP 192.168.0.69.

Edit: Fixed format

[christophrothmann]

@remy415
Yeah, thanks for your comment but the issue that I have is, that I make that API call from a Chrome Browser Extension.

I have written a browser extension and when I click on the button of that little window, I make an API call to my local hosted Ollama instance on my pc

Here my Plugin:
--removed the image

...and my api call:

...and the required (I believe)
permissions in the manifest.json File

Maybe that information helps :)

[christophrothmann]

I also set the OLLAMA_HOST=0.0.0.0

[remy415]

Can you confirm the API is working properly with a curl request from cli? curl http://localhost:11434/api/generate -d '{ "model": "codellama", "prompt": "Why is the sky blue?", "stream": false }' Edit: Removed metadata sent by my mail client

[christophrothmann]

Hi @remy415 ,
yes I confirm that. I tested it with Postman. 😎😉
Maybe Ollama allows no request from an browser extension 😅.

Just to clarify, I did not clone the Ollama repo itself and run that. I run the Application downloaded from the official ollama.ai webpage. Should I try to clone and run that? Makes that a difference?

Best,
Chris

[remy415]

@DevChrisRoth
Building from src likely won't change the issue you're having, that's mostly just to change architectures or enable different CPU features (AVX512, etc) for the llama_cpp backend.

Can you enable debug on startup with OLLAMA_DEBUG=1 ollama serve and try the connect again, check your screen for the api requests and see what the server logs say when you connect

[remy415]

Also, some people on the below stackoverflow forum had problems with other extensions interacting with theirs. Try disabling all other extensions and see if that fixes it.

In my case it was done due to an extension that I had on Chrome, try disabling all extensions and then try again. This may help someone else. I was stuck 3 days on this one tried everything and in the end the issue was caused by extension... –
nikola3103
Feb 15, 2022 at 10:07

[remy415]

Have you double-checked that your extension code is good? Is your button sending the proper messages to trigger the extension eventListener?

[christophrothmann]

Hey @remy415 ,
I started the ollama serve with your command and got following logging messages back:

I have not activated any other browser extensions, but thanks for your advice :)

[mxyng]

Extensions have their own origin. For Chrome, this should be chrome-extension:// so you'll need this for OLLAMA_ORIGINS

OLLAMA_ORIGINS=chrome-extension://...

See gin-contrib/cors

[christophrothmann]

@mxyng ,
so I have to execute following in the terminal? (I'm using it on mac)
launchctl setenv OLLAMA_ORIGIN "chrome-extension://" and then run ollama serve?

Sorry to ask such a stupid question, but I'm new to this :)

[tomjorquera]

I have the same issue, due to CORS (everything work fine when using non-browser clients).

@DevChrisRoth I worked around it for now by adding the following in my service file:

[Service]
ExecStart=...
Environment="OLLAMA_ORIGINS=moz-extension://*,chrome-extension://*"

It allows me to test my extension locally, but seems very cumbersome, as each user would need to configure their ollama instance with a similar setting.

What I don't understand is that why this behavior occurs despite me setting the proper host_permissions and calling from a background script. This is supposed to allow me to "access to those origins without cross-origin restrictions" (as per the doc linked).

I tried both with the host_permissions of manifest V3 and the old permissions setting of V2, but still get this behavior both in Firefox and Chromium.

[christophrothmann]

Hell yeah,
I finally figured it out.
I looked into the server.log file from Ollama ~/.ollama/logs/server.log and there was a useful error message.

The solution was to start the Ollama Service in the terminal with following command:
OLLAMA_ORIGINS=chrome-extension://* ollama serve

Thanks for your help guys! @mxyng , @remy415 , @tomjorquera