Running on Debian ARM64 Hardware HOWTO

[Martyn575]

Background ARM64 System And OS:

I am running a Radxa Orion o6 64GB of ram. Running Debian 12 - customised by Radxa with hardware acceleration.

Ollama is already installed on this system and working well.

I am leaving this here because it might be useful to someone else trying to get this working. The steps are a bit different on ARM64 hardware running Debian than on x64 systems running Debian - where stuff works out of the box.

Since less people run ARM64 systems than x64, I figured i'd share how i got it working.

Apologies if this is posted in the wrong place. Q&A seemed like the best place for this.

Installation Steps That Worked For Me:

Firstly install docker:

curl -fsSL https://get.docker.com | sudo sh

Docker compose build won't run for me unless this is set:

cat /etc/docker/daemon.json
{
  "features": {
    "containerd-snapshotter": false
  }
}

Next run the main steps that download and setup Odysseus:

sudo bash
mkdir -p /opt/Odysseus
cd /opt/Odysseus
git clone https://github.com/pewdiepie-archdaemon/odysseus.git
cd odysseus

We need to edit the docker compose so that that web backend listens to remote connections elsewhere on the lan, this involves changing the following two lines in the docker-compose.yml

cat docker-compose.yml
...
    ports:
      - "${APP_BIND:-0.0.0.0}:${APP_PORT:-7000}:7000"
...
      - ALLOWED_ORIGINS=*
...

Note: leave it as is if you do not want to share the page with the rest of the LAN (why wouldn't you?)

Duplicate the env file and customise it to have a username and password (Note: on ARM64 it doesn't generate this automatically in the log for some reason - it did with my x86 but not on ARM, I don't know why).

Edit the .env file to end with the following two lines:

cp .env.example .env
cat .env
....
ODYSSEUS_ADMIN_USER=admin
ODYSSEUS_ADMIN_PASSWORD=Password1

Now, if you have ollama running, you need to set the thing to accept all accept inbound connections from Odysseus:

systemctl edit ollama.service

It must contain the lines:

[Service]
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_HOST=0.0.0.0"

After modifying this, save it, and reload and restart the ollama service:

systemctl daemon-reload
systemctl restart ollama

Now, run ollama list, and check you have a model installed. If you do not, then install one:

ollama pull gemma3:27b

At this point, you are ready to start Odysseus:

docker compose up -d --build

This will build it. Once it builds it will probably take a couple of minutes to provide a login.

Odysseus Quickstart Configuration:

1. Navigate to 192.168.1.40:7000 (or whatever your AI system is), and login using "admin" / "Password1".
2. Navigate in the Odysseus GUI and change your password. This is the cog icon on the bottom left. Navigate to Account.
3. Next in the same settings menu, navigate to: "Add Models".
4. In the Local Models, add LLM and http://172.17.0.1:11434 and click "add". It should appear with at least one model.
5. Close settings
6. On the left hand side panel click "+ New Chat" and then on the bottom right above the text input box select the model and then enter a query to test it works.

Odysseus With Image Processing

In the chat you can attach images to be read by models such as qwen3-vl:latest. These are useful for interpretting images - and pretty useful for screen scraping.

However, odysseus times out before the llm can respond we can change the default timeout as follows:

cd /opt/Odysseus/odysseus/src

Then edit the file: ai_interaction.py and change the line that says:

AI_CHAT_TIMEOUT = 120  # seconds for a single LLM call

To:

AI_CHAT_TIMEOUT = 600  # seconds for a single LLM call

and in llm_core.py change the line DEFAULT_TIMEOUT = 30 to DEFAULT_TIMEOUT = 600

Then to restart Odysseus we need to rebuild the docking container and clear some stuff out, otherwise it'll seem to work and then revert back next reboot.

cd ..
docker compose down --volumes --remove-orphans
docker builder prune -f
docker compose build --no-cache
docker compose up -d