Skip to content

Add AWF /reflect reference for gateway-based model discovery and routing#31540

Merged
pelikhan merged 3 commits into
mainfrom
copilot/add-documentation-page-for-reflect-route
May 11, 2026
Merged

Add AWF /reflect reference for gateway-based model discovery and routing#31540
pelikhan merged 3 commits into
mainfrom
copilot/add-documentation-page-for-reflect-route

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 11, 2026

New Feature

This PR adds a dedicated reference page for AWF GET /reflect, documenting how shared workflows/tools discover gateway inference endpoints, determine model availability, and choose models dynamically at runtime. It also codifies the requirement to route inference through the gateway rather than direct provider endpoints to preserve cost controls, tracking, and optimization signals.

  • What does this feature do?

    • Adds docs/src/content/docs/reference/awf-reflect.md as the canonical /reflect documentation.
    • Documents response fields needed for runtime routing (provider, base_url, configured, models, models_url, models_fetch_complete).
    • Defines a practical selection flow: query /reflect → filter configured endpoints → prefer populated model lists → select provider/model dynamically.
    • Adds explicit gateway-only inference guidance for shared components.
  • Why is this feature needed?

    • Shared workflows/tools run across environments with different provider configuration and model availability.
    • Hardcoded direct model endpoints break portability and bypass gateway-level governance/observability.
  • Implementation details

    • Adds sidebar navigation entry in docs/astro.config.mjs under Reference (/reference/awf-reflect/).
    • Includes bounded retry guidance for transient models: null states during model discovery.
curl -s http://api-proxy:10000/reflect
Selection pattern:
1) GET /reflect
2) endpoints where configured == true
3) prefer endpoints with non-empty models
4) match requested alias/pattern against models
5) call selected endpoint.base_url (via gateway only)

Copilot AI and others added 3 commits May 11, 2026 16:22
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Add AWF /reflect route documentation page Add AWF /reflect reference for gateway-based model discovery and routing May 11, 2026
Copilot AI requested a review from pelikhan May 11, 2026 16:30
@pelikhan pelikhan marked this pull request as ready for review May 11, 2026 16:53
Copilot AI review requested due to automatic review settings May 11, 2026 16:53
@pelikhan pelikhan merged commit 38998ad into main May 11, 2026
@pelikhan pelikhan deleted the copilot/add-documentation-page-for-reflect-route branch May 11, 2026 16:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new documentation reference page for AWF GET /reflect, intended to guide shared workflows/tools in discovering gateway inference endpoints, understanding model availability, and routing requests through the gateway.

Changes:

  • Adds a new reference doc page: docs/src/content/docs/reference/awf-reflect.md.
  • Documents recommended runtime selection flow and retry guidance for transient model discovery states.
  • Adds a new sidebar entry under Reference in docs/astro.config.mjs.
Show a summary per file
File Description
docs/src/content/docs/reference/awf-reflect.md New /reflect reference page describing endpoint/model discovery and dynamic routing guidance.
docs/astro.config.mjs Adds the new AWF Reflect Route page to the Reference sidebar navigation.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

docs/src/content/docs/reference/awf-reflect.md:43

  • Step 6 suggests retrying discovery whenever models is null, but in practice models can be null for reasons that won’t change with retries (e.g., models_url is null, or model listing isn’t supported), and the more reliable signal for “still warming up” is models_fetch_complete plus whether models_url is present for a configured endpoint. Suggest adjusting the guidance to (a) only retry /reflect while models_fetch_complete is false, and/or (b) if configured is true and models_url is present, query models_url directly to populate models; otherwise treat models as unavailable.
1. Query `/reflect` at start of execution.
2. Filter endpoints to `configured: true`.
3. Prefer endpoints with a non-empty `models` list.
4. Match requested model aliases/patterns against available models.
5. Route inference to the selected endpoint `base_url`.
6. If `models` is `null`, retry discovery with bounded backoff (for example, every 3 seconds up to 5 attempts) before failing.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Comment on lines +30 to +42
- `endpoints[].configured`: whether credentials/config are present for that provider
- `endpoints[].models`: discovered model IDs, or `null` when model discovery is not yet complete
- `endpoints[].models_url`: gateway URL used to query models for that provider
- `models_fetch_complete`: whether startup model discovery is complete

## Recommended selection flow for shared tools

1. Query `/reflect` at start of execution.
2. Filter endpoints to `configured: true`.
3. Prefer endpoints with a non-empty `models` list.
4. Match requested model aliases/patterns against available models.
5. Route inference to the selected endpoint `base_url`.
6. If `models` is `null`, retry discovery with bounded backoff (for example, every 3 seconds up to 5 attempts) before failing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants