Add local inference server auto-detection and improve stack setup#313
Closed
bussyjd wants to merge 1 commit into
Closed
Add local inference server auto-detection and improve stack setup#313bussyjd wants to merge 1 commit into
bussyjd wants to merge 1 commit into
Conversation
Resolve conflicts in CLAUDE.md (take expanded package table and deps from main) and internal/dns/resolver.go (take quiet error style from main, removing verbose print instructions). https://claude.ai/code/session_01DuimxJRYMS1KLkyURM3yYp
Collaborator
Author
|
Stale — #299 has been merged into feat/monetize-path directly. This merge PR is no longer needed. |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds automatic discovery of local inference servers (llama-server, ollama, vLLM, LiteLLM) to streamline the
obol sell inferenceworkflow, improves the Obol Stack setup experience with better prerequisite checking and DNS configuration, and fixes several operational issues around file permissions and Ollama configuration.Key Changes
Inference Server Auto-Detection
internal/inference/detect.go: Implements concurrent probing of well-known inference server ports (8080, 11434, 8000, 4000) with OpenAI-compatible/v1/modelsendpoint enumerationProbeEndpoint()/ProbeEndpointContext(): Probe a single host:port for inference serversScanLocalEndpoints()/ScanLocalEndpointsContext(): Concurrently scan all common portsDetectServerType(): Identify server software (ollama, llama-server, openai-compat) with priority-ordered detectionFormatEndpointDisplay(): Pretty-print discovered endpoints and their modelsinternal/inference/detect_test.goincluding happy path, malformed JSON, context cancellation, and error casesCLI Integration
cmd/obol/sell.go: Auto-detect and prompt user to select model/upstream when--modelflag is not provided and running interactively--modelflag in non-interactive modeStack Setup & Prerequisites
obolup.sh:check_prerequisites()function validates Node.js 22+ / npm availability before installationconfigure_ollama_host_binding()to setOLLAMA_HOST=0.0.0.0on Linux systemd for k3d container accessOBOL_BIN_DIRis in PATH during dependency installation for inter-tool availabilityDNS & Networking
internal/stack/stack.go:wildcardDNSparameter toUp()function for opt-in wildcard DNS configuration/etc/hostsentries forobol.stack(works everywhere)*.obol.stackDNS via NetworkManager/dnsmasq (Linux) or/etc/resolver(macOS) is now opt-in via--wildcard-dnsflaginternal/dns/resolver.go: Made Linux resolver configuration non-fatal, allowing fallback to/etc/hostson headless servers without NetworkManagerFile Permissions & Volume Management
internal/openclaw/openclaw.go:fixVolumeOwnership()function normalizes PVC file ownership for container access (UID 1000)internal/embed/infrastructure/base/templates/local-path.yaml: Changedchowntochown -Rfor recursive ownership fixesBuild & Deployment
.github/workflows/docker-publish-x402.yml: Refactored to build both x402-verifier and x402-buyer images in parallel using matrix strategyrenovate.json: Added custom regex rules for obolup.sh tool version updates and k3s image tag updates with proper grouping and schedulingDocumentation
CLAUDE.md: Updated dependency documentation with structured table of packages, key files, and rolesNotable Implementation Details
https://claude.ai/code/session_01DuimxJRYMS1KLkyURM3yYp