How to get an LLM to use in odysseus

[MunMun06]

Im an AI noob that just use the web ui to talk to LLM. I dont have any api keys for LLM. How do i get one

[DanteTheRA]

ok so go to open router. create a api key. Save it somewhere so you don't loose it. (it wont tell you again so really do save it somewhere) put it in the api key for odysseus and you have an ai. now normally youd have to pay as you go. however if you use nemotron 3 super (free) its well... free. you could also self host if thats an option for you. nemotron 3 super is a powerful model good at agentic tasks as well as most other things so its a solid way to start. if you need more instruction on how to get an api key im sure someone can help you. I have to get ready for work and do somethings. if i could play with this project all day i would.

[MunMun06]

I want to use LLM from provider and not local hosting. What are the provider that are free? Or i should actually local host the LLM instead?

[CoolJohn-lab]

You should definitely local host the LLM. This app will leak all your personal information to the highest bidder if you use it on a free API. The only reason those free APIs exist is that they are hoping someone will type in something valuable about themselves to their free prompt, that they can go sell to a data broker.

[Osman-bin-nasir]

To host the llms locally you could just Install Ollama - https://ollama.com/

Pull a model (choose based on your system specs)
For example:
ollama run llama3 or smaller models if your PC isn’t very powerful.

Once it’s running, Ollama automatically exposes an API locally.

In Odysseus, just set your endpoint URL to: http://127.0.0.1:11434/

now Odysseus will use your local model

[DanteTheRA]

I’m wondering because the app wasn’t seeing both my gpus when trying to scan for models that would be good to run locally. If I run a model locally how do I know it will use the stronger gpu? Or will it do it automatically even though the software doesn’t recognize it. Don’t want to overload my bad gpu with something it can’t run. And I’ll need to find out later which local Ai I’m even capable of running myself

[JNLN01]

Is it necessary to run Ollama separately in the terminal and then open Odysseus, or does Odysseus launch Ollama upon startup?

[BarsatZulkarnine]

If you do not want to self-host, OpenRouter is probably your best bet. Free API key, no billing required to start, and you get 50 requests/day on free models (goes up to 1,000/day if you add a small amount of credits). More details in this thread.

[CoolJohn-lab]

Odyssey is designed to create thousands of prompts per day, churning away on tasks in the background. Hooking it up to an LLM API is a good way to use up every credit you have on nothing. It's also a good way to get hacked if you use the email client.

[bitboody]

The way I did it was I downloaded Ollama and I choose the first model the cookbook feature recommended to me.

I run it in the background in the terminal and then Odysseus detects it using the default settings.

[ravijaanthony]

Configure Ollama to Allow Docker (Linux)

If you prefer to keep your Docker container isolated, you must change Ollama's background service configuration so it listens to the Docker bridge, and also tell it to allow outside requests (CORS).

1.Open the service configuration:Requires sudo.
Run this in your terminal to open the override file in your default text editor:
sudo systemctl edit ollama.service

2.Add the environment variables:Look for the space between the comment blocks. Add these exact lines to tell Ollama to listen to all IPs and allow cross-origin requests:Ini, TOML

[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=*"

3.Apply the changes and restart:Save the file and exit the editor (if using nano, press Ctrl+O, Enter, then Ctrl+X). Finally, restart the service by running:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Once the service restarts, go back into Odysseus and try http://172.17.0.1:11434 again. And then add the url in odysseus as a local model and enjoy.

I used gemini to explain and this is what i got