beps: add BEP-0015 AI Model Provider Service#33906
Open
pguppy wants to merge 4 commits into
Open
Conversation
As dicussed in SIG meeting, I have made this into a new core service using multiton to ensure easy extendability for maintainers to expand with there own model providers. Signed-off-by: Patrick Guppy <171629661+pguppy@users.noreply.github.com>
backstage-goalie
Bot
added
size:large
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new BEP proposing a core backend AI model provider service for Backstage, including the service contract shape, provider factory extensibility via multiton service refs, and an operator-facing configuration model.
Changes:
- Introduces BEP 0014 describing a service-first, provider-agnostic AI model access layer for backend consumers.
- Specifies an extensibility mechanism for provider families (multiton service ref contributions) and a consolidated
ai.providers.*configuration approach. - Outlines capability gating, startup validation expectations, and an initial provider packaging strategy (plus future HTTP facade considerations).
Corrected typos and improved clarity in the README. Signed-off-by: Patrick Guppy <171629661+pguppy@users.noreply.github.com>
pguppy
changed the title
pguppy
changed the title
- Move beps/0014-ai-provider-service/ to beps/0015-ai-provider-service/ since 0014 is already taken by beps/0014-ai-service/ - Add 'Mastra' and 'runtimes' to Vale vocabulary accept.txt to fix failing Verify Docs Quality workflow (10 spelling errors) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Patrick Guppy <patrick.guppy@bunnings.com.au>
- Fix H1 heading to match frontmatter title: 'AI model provider service' - Fix typo: 'provder' -> 'provider' in summary - Nest Goals/Non-Goals under Motivation in TOC - Capitalize section headings to match BEP template: Non-goals -> Non-Goals, Design details -> Design Details, Release plan -> Release Plan - Add StreamTextResponse interface with textStream iterable + response promise for finish reason and token usage metadata, replacing bare AsyncIterable<string> return type - Fix duplicate 'anthropic' YAML key in example config; rename second instance to 'anthropicInternal' Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Patrick Guppy <patrick.guppy@bunnings.com.au>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
beps: BEP-0015 — AI Model Provider Service
This BEP proposes a new core AI model provider service for Backstage, a provider-agnostic, framework-level service that gives backend plugins a stable contract for accessing AI model capabilities without wiring their own SDKs, credentials, or config.
The core problem: there is no standard way for backend features to consume AI capabilities today. Plugin authors that want to add AI must choose a specific provider, wire authentication themselves, define their own config shape, and handle provider-specific error handling and model selection. This leads to repeated implementation across plugins, fragmented credentials, and vendor lock-in at the wrong layer.
The proposal introduces a
coreServices.aiModelsbackend service backed by AI SDK as the provider transport layer, with an extensibleaiProviderFactoriesServiceRef(multiton) for contributing provider families. All provider configuration lives under a singleai.providers.*key inapp-config.yaml. Provider identity follows the request protocol and auth model, not the model vendor name.Key design decisions:
AIModelsServiceis the Backstage-owned public contract — consumers use Backstage request/response types, not raw AI SDK typescreateServiceFactoryusing the multitonaiProviderFactoriesServiceRef, keeping extensibility at the framework level rather than plugin extension points@backstage/backend-defaults, following the same pattern as built-in URL readersresolveModel()method provides AI SDK-compatible model objects for advanced runtime interop with tools like Mastra, without making that the primary usage patternLooking for feedback on the overall direction, the provider family contribution model, and any gaps in the service surface.
✔️ Checklist
Signed-off-byline in the message. (more info)