Skip to content

Add self-hosted local LLM provider support#2

Merged
manueljpconde merged 1 commit into
mainfrom
codex/local-llm-provider
May 9, 2026
Merged

Add self-hosted local LLM provider support#2
manueljpconde merged 1 commit into
mainfrom
codex/local-llm-provider

Conversation

@manueljpconde
Copy link
Copy Markdown
Owner

Summary

  • Adds a server-configured local:server LLM provider backed by OpenAI-compatible Chat Completions streaming.
  • Exposes backend-derived local provider availability/capabilities without exposing local endpoint secrets.
  • Gates local tool usage unless LOCAL_LLM_SUPPORTS_TOOLS=true, and keeps local out of tabular preferences unless tool support is enabled.
  • Adds backend tests for provider routing, local config parsing, SSE parsing, and no-tool local streaming.
  • Documents Ollama and LM Studio self-hosting examples.

Verification

  • npm test --prefix backend
  • npm run build --prefix backend
  • npx eslint 'src/app/(pages)/account/models/page.tsx' 'src/app/components/assistant/ModelToggle.tsx' 'src/app/lib/modelAvailability.ts' 'src/contexts/UserProfileContext.tsx' 'src/app/lib/mikeApi.ts' 'src/app/components/shared/ApiKeyMissingModal.tsx' from frontend/
  • NEXT_PUBLIC_SUPABASE_URL=https://example.supabase.co NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY=placeholder npm run build --prefix frontend

Note: full npm run lint --prefix frontend still fails on pre-existing lint issues outside this change set.

Copy link
Copy Markdown
Owner Author

@manueljpconde manueljpconde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@manueljpconde manueljpconde marked this pull request as ready for review May 9, 2026 11:57
@manueljpconde manueljpconde merged commit c8ef360 into main May 9, 2026
@manueljpconde manueljpconde deleted the codex/local-llm-provider branch May 9, 2026 11:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant