Skip to content

Wildcard endpoint proxying + OpenRouter-style provider routing#288

Merged
ericflo merged 7 commits intomainfrom
ce/wildcard-provider-routing
Apr 15, 2026
Merged

Wildcard endpoint proxying + OpenRouter-style provider routing#288
ericflo merged 7 commits intomainfrom
ce/wildcard-provider-routing

Conversation

@ericflo
Copy link
Copy Markdown
Owner

@ericflo ericflo commented Apr 15, 2026

Summary

Replace the three hardcoded POST route handlers (/v1/chat/completions, /v1/messages, /v1/responses) with a single wildcard catch-all handler that proxies any HTTP method and path to workers. Adds OpenRouter-compatible provider routing preferences so clients can express provider ordering, inclusion/exclusion, and fallback behavior.

What changed

Protocol (modelrelay-protocol)

  • RequestMessage gains method: String (defaults to "POST" for backward compat) and endpoint_path already existed
  • RegisterMessage / RegisterAck gain endpoint_prefixes: Vec<String> so workers advertise which paths they handle
  • New ProviderRouting struct with only, ignore, order, allow_fallbacks, sort fields

Server core (modelrelay-server/src/lib.rs)

  • WorkerState tracks endpoint_prefixes and exposes supports_endpoint() for prefix matching
  • submit_transport_request accepts endpoint_path, method, and Option<ProviderRouting>
  • Worker selection respects endpoint capability and provider routing preferences
  • find_eligible_worker_id_with_routing implements the full routing algorithm

HTTP layer (modelrelay-server/src/http.rs)

  • Removed chat_completions_handler, messages_handler, responses_handler
  • Added wildcard_handler as .fallback() — extracts method, full path, headers, body
  • Model extracted from JSON body model field or X-Model header
  • Provider routing from body provider field or X-Provider header (JSON)
  • All HTTP methods supported (GET/POST/PUT/DELETE/PATCH/HEAD/OPTIONS)

Worker (modelrelay-worker)

  • WorkerDaemonConfig gains endpoint_prefixes field
  • CLI/env/config-file support for --endpoint-prefixes / ENDPOINT_PREFIXES
  • forward_request uses dynamic method dispatch instead of hardcoded .post()

Backward compatibility

  • All new protocol fields use #[serde(default)] / skip_serializing_if — existing workers/clients work unchanged
  • method defaults to "POST", endpoint_prefixes defaults to empty (accept all)
  • Provider routing is entirely optional

Test plan

  • All 256 existing tests pass across protocol, server, worker, contract-tests, cloud, and web crates
  • Manual test with worker registering endpoint prefixes
  • Manual test with provider routing headers

🤖 Generated with Claude Code

Cloud Eric added 5 commits April 15, 2026 17:09
- Add endpoint_prefixes to WorkerState and AdminWorkerInfo
- Add method and provider_routing to RequestRecord/RequestTransport
- Update submit_transport_request with method + provider_routing args
- Implement provider routing in worker selection (only/ignore/order/fallbacks)
- Replace 3 hardcoded POST routes with single catch-all wildcard handler
- Support all HTTP methods (GET/POST/PUT/DELETE/PATCH)
- Extract model from JSON body or X-Model header
- Extract provider routing from body or X-Provider header
- Add endpoint_prefixes field to WorkerDaemonConfig
- Add --endpoint-prefixes CLI flag and ENDPOINT_PREFIXES env var
- Add endpoint_prefixes to FileConfig for TOML support
- Include endpoint_prefixes in RegisterMessage sent to server
- Use request method field for backend forwarding (all HTTP methods)
@ericflo ericflo force-pushed the ce/wildcard-provider-routing branch from b1f7f67 to 86fc12c Compare April 15, 2026 17:11
@ericflo ericflo merged commit 564e092 into main Apr 15, 2026
12 checks passed
@ericflo ericflo deleted the ce/wildcard-provider-routing branch April 15, 2026 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant