Skip to content

[chore] Set up a self-hosted GitHub Actions runner on the Ollama network #22

@dogkeeper886

Description

@dogkeeper886

Context

FR-005 landed (closed by #19 / PR #21). The workflows now consume LLM_JUDGE_URL / LLM_JUDGE_MODEL / TL_DEV_KEY from repo secrets/vars, but all workflows still target `runs-on: ubuntu-latest`. Dual-mode LLM judging from a GitHub-hosted runner can't reach an Ollama instance on a private LAN without exposing that endpoint publicly.

FR-005's Proposed solution option 1 — a self-hosted runner on the same network as the existing Ollama instance at `192.168.2.103:11434` — is the path this ticket picks up.

Must

  • Register a self-hosted runner on a box with network access to the Ollama instance (systemd service or equivalent so it survives reboots).
  • Pick a label scheme (e.g. `self-hosted, linux, llm`) and document it.
  • Switch `runs-on:` in `.github/workflows/test-suite.yml` to the self-hosted label (probably via an input or a variable so operators can still dispatch against `ubuntu-latest` when they want).
  • Populate `secrets.LLM_JUDGE_URL` with the LAN address.
  • Dispatch `test-pipeline.yml` with `judge_mode=dual` end-to-end on the self-hosted runner and confirm it hits Ollama cleanly.
  • Extend `cicd/CI_SETUP.md` with a short runner-setup section (registration, labels, restart policy).

Nice to have

  • Auto-update the runner agent. GitHub pushes runner updates regularly; staleness can break dispatches.
  • A manual `ubuntu-latest` fallback path so the suite can still be dispatched when the self-hosted runner is down.

Out of scope

  • Switching to a hosted LLM API — separate decision, separate ticket.
  • Re-enabling automatic push/PR triggers — downstream of this, decide once the self-hosted path is stable.

Acceptance criteria

  • A dispatch of `test-pipeline.yml` with `judge_mode=dual` on the self-hosted runner completes with LLM judge reaching the LAN Ollama instance and scoring normally.
  • Runner survives a host reboot without manual intervention.
  • `cicd/CI_SETUP.md` names the runner's expected labels, restart policy, and operator checklist.

Blocks / unblocks

  • Unblocks the follow-up decision on re-enabling push/PR triggers (FR-005 open question).
  • Does not block the hosted-API-alternative ticket — they solve the same problem differently, pick whichever lands first.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions