Skip to content

extension/llm/server: serving foundations (schemas, errors, templating, tools)#19993

Open
mergennachin wants to merge 10 commits into
gh/mergennachin/2/headfrom
gh/mergennachin/4/head
Open

extension/llm/server: serving foundations (schemas, errors, templating, tools)#19993
mergennachin wants to merge 10 commits into
gh/mergennachin/2/headfrom
gh/mergennachin/4/head

Conversation

@mergennachin

@mergennachin mergennachin commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Add the OpenAI server's standalone control-plane building blocks, independent of
the HTTP layer and of any model runtime: OpenAI request/response schemas
(protocol.py), structured errors (errors.py), HF chat templating
(chat_template.py), and tool-call parsers (tool_parsers/) for Hermes-style JSON
and Qwen XML. The wire contract is documented in spec/README.md. Unit-tested
under tests/ (tool parsing).

The HTTP server builds on these.

Part of #20001

[ghstack-poisoned]
[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jun 3, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19993

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 1842447 with merge base eeb0646 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
[ghstack-poisoned]
@mergennachin mergennachin changed the base branch from gh/mergennachin/3/head to gh/mergennachin/2/head June 4, 2026 22:14
[ghstack-poisoned]
[ghstack-poisoned]
@mergennachin mergennachin marked this pull request as ready for review June 5, 2026 18:56
[ghstack-poisoned]
@mergennachin mergennachin changed the title extension/llm/server: serving foundations (schemas, tool parser, prefix cache) extension/llm/server: serving foundations (schemas, errors, templating, tools) Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant