Skip to content

feat: Implement FastAPI router system #4191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

feat: Implement FastAPI router system #4191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
eb3cab1
feat: Implement FastAPI router system
leseb Nov 19, 2025
2fe24a6
chore: move ListBatchesResponse to models.py
leseb Nov 20, 2025
00e7ea6
fix: adopt FastAPI directly in llama-stack-api
leseb Nov 20, 2025
30cab02
chore: refactor Batches protocol to use request models
leseb Nov 20, 2025
2003042
chore: same as previous commit but for more fields
leseb Nov 20, 2025
9595619
chore: remove empty dir
leseb Nov 20, 2025
f62c604
chore: update unit test to use previously created Class
leseb Nov 20, 2025
23e7444
chore: rename routes.py to fastapi_routes.py
leseb Nov 21, 2025
8a21d8d
chore: mv router_registry.py to fastapi_router_registry.py
leseb Nov 21, 2025
95e9455
chore: removed impl_getter from router function
leseb Nov 21, 2025
234eaf4
chore: remove impl_getter function
leseb Nov 21, 2025
6f552e0
fix: mypy
leseb Nov 21, 2025
ac816a6
fix: move models.py to top-level init
leseb Nov 21, 2025
03a3126
chore: more accurate route parcing
leseb Nov 24, 2025
49005f1
fix: use hardcoded list and dictionary mapping for router registry
leseb Nov 24, 2025
87e60bc
chore: move dep functions outside of create_router
leseb Nov 24, 2025
4f08a62
chore: remove telemetry code for routers
leseb Nov 24, 2025
a6aaf18
chore: generate FastAPI dependency functions from Pydantic models to …
leseb Nov 24, 2025
6d76a63
fix: mypy
leseb Nov 24, 2025
3770963
Merge branch 'main' into routeur
leseb Nov 24, 2025
9a2b4ef
chore: clarify function and log about which router
leseb Nov 25, 2025
b0b3034
chore: rm leftover
leseb Nov 25, 2025
3dc5b5d
fix: more accurate type
leseb Nov 25, 2025
ead9e63
fix: no inline import
leseb Nov 25, 2025
f330c8e
chore: simplify route addition when calling inspect
leseb Nov 25, 2025
9534293
Merge branch 'main' into routeur
leseb Nov 25, 2025
98f202b
Merge branch 'main' into routeur
leseb Nov 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 68 additions & 14 deletions client-sdks/stainless/openapi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ paths:
description: Default Response
tags:
- Batches
summary: List Batches
summary: List all batches for the current user.
description: List all batches for the current user.
operationId: list_batches_v1_batches_get
parameters:
Expand All @@ -48,14 +48,18 @@ paths:
anyOf:
- type: string
- type: 'null'
description: Optional cursor for pagination. Returns batches after this ID.
title: After
description: Optional cursor for pagination. Returns batches after this ID.
- name: limit
in: query
required: false
schema:
type: integer
description: Maximum number of batches to return. Defaults to 20.
default: 20
title: Limit
description: Maximum number of batches to return. Defaults to 20.
post:
responses:
'200':
Expand All @@ -76,9 +80,11 @@ paths:
default:
$ref: '#/components/responses/DefaultError'
description: Default Response
'409':
description: 'Conflict: The idempotency key was previously used with different parameters.'
tags:
- Batches
summary: Create Batch
summary: Create a new batch for processing multiple API requests.
description: Create a new batch for processing multiple API requests.
operationId: create_batch_v1_batches_post
requestBody:
Expand All @@ -97,20 +103,20 @@ paths:
schema:
$ref: '#/components/schemas/Batch'
'400':
description: Bad Request
$ref: '#/components/responses/BadRequest400'
description: Bad Request
'429':
description: Too Many Requests
$ref: '#/components/responses/TooManyRequests429'
description: Too Many Requests
'500':
description: Internal Server Error
$ref: '#/components/responses/InternalServerError500'
description: Internal Server Error
default:
description: Default Response
$ref: '#/components/responses/DefaultError'
description: Default Response
tags:
- Batches
summary: Retrieve Batch
summary: Retrieve information about a specific batch.
description: Retrieve information about a specific batch.
operationId: retrieve_batch_v1_batches__batch_id__get
parameters:
Expand All @@ -119,7 +125,9 @@ paths:
required: true
schema:
type: string
description: 'Path parameter: batch_id'
description: The ID of the batch to retrieve.
title: Batch Id
description: The ID of the batch to retrieve.
/v1/batches/{batch_id}/cancel:
post:
responses:
Expand All @@ -130,20 +138,20 @@ paths:
schema:
$ref: '#/components/schemas/Batch'
'400':
description: Bad Request
$ref: '#/components/responses/BadRequest400'
description: Bad Request
'429':
description: Too Many Requests
$ref: '#/components/responses/TooManyRequests429'
description: Too Many Requests
'500':
description: Internal Server Error
$ref: '#/components/responses/InternalServerError500'
description: Internal Server Error
default:
description: Default Response
$ref: '#/components/responses/DefaultError'
description: Default Response
tags:
- Batches
summary: Cancel Batch
summary: Cancel a batch that is in progress.
description: Cancel a batch that is in progress.
operationId: cancel_batch_v1_batches__batch_id__cancel_post
parameters:
Expand All @@ -152,7 +160,9 @@ paths:
required: true
schema:
type: string
description: 'Path parameter: batch_id'
description: The ID of the batch to cancel.
title: Batch Id
description: The ID of the batch to cancel.
/v1/chat/completions:
get:
responses:
Expand Down Expand Up @@ -3950,29 +3960,35 @@ components:
input_file_id:
type: string
title: Input File Id
description: The ID of an uploaded file containing requests for the batch.
endpoint:
type: string
title: Endpoint
description: The endpoint to be used for all requests in the batch.
completion_window:
type: string
const: 24h
title: Completion Window
description: The time window within which the batch should be processed.
metadata:
anyOf:
- additionalProperties:
type: string
type: object
- type: 'null'
description: Optional metadata for the batch.
idempotency_key:
anyOf:
- type: string
- type: 'null'
description: Optional idempotency key. When provided, enables idempotent behavior.
type: object
required:
- input_file_id
- endpoint
- completion_window
title: CreateBatchRequest
description: Request model for creating a batch.
Batch:
properties:
id:
Expand Down Expand Up @@ -12760,6 +12776,44 @@ components:
- query
title: VectorStoreSearchRequest
type: object
ListBatchesRequest:
description: Request model for listing batches.
properties:
after:
anyOf:
- type: string
- type: 'null'
description: Optional cursor for pagination. Returns batches after this ID.
nullable: true
limit:
default: 20
description: Maximum number of batches to return. Defaults to 20.
title: Limit
type: integer
title: ListBatchesRequest
type: object
RetrieveBatchRequest:
description: Request model for retrieving a batch.
properties:
batch_id:
description: The ID of the batch to retrieve.
title: Batch Id
type: string
required:
- batch_id
title: RetrieveBatchRequest
type: object
CancelBatchRequest:
description: Request model for canceling a batch.
properties:
batch_id:
description: The ID of the batch to cancel.
title: Batch Id
type: string
required:
- batch_id
title: CancelBatchRequest
type: object
DialogType:
description: Parameter type for dialog data with semantic output labels.
properties:
Expand Down
44 changes: 44 additions & 0 deletions docs/static/deprecated-llama-stack-spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -793,29 +793,35 @@ components:
input_file_id:
type: string
title: Input File Id
description: The ID of an uploaded file containing requests for the batch.
endpoint:
type: string
title: Endpoint
description: The endpoint to be used for all requests in the batch.
completion_window:
type: string
const: 24h
title: Completion Window
description: The time window within which the batch should be processed.
metadata:
anyOf:
- additionalProperties:
type: string
type: object
- type: 'null'
description: Optional metadata for the batch.
idempotency_key:
anyOf:
- type: string
- type: 'null'
description: Optional idempotency key. When provided, enables idempotent behavior.
type: object
required:
- input_file_id
- endpoint
- completion_window
title: CreateBatchRequest
description: Request model for creating a batch.
Batch:
properties:
id:
Expand Down Expand Up @@ -9603,6 +9609,44 @@ components:
- query
title: VectorStoreSearchRequest
type: object
ListBatchesRequest:
description: Request model for listing batches.
properties:
after:
anyOf:
- type: string
- type: 'null'
description: Optional cursor for pagination. Returns batches after this ID.
nullable: true
limit:
default: 20
description: Maximum number of batches to return. Defaults to 20.
title: Limit
type: integer
title: ListBatchesRequest
type: object
RetrieveBatchRequest:
description: Request model for retrieving a batch.
properties:
batch_id:
description: The ID of the batch to retrieve.
title: Batch Id
type: string
required:
- batch_id
title: RetrieveBatchRequest
type: object
CancelBatchRequest:
description: Request model for canceling a batch.
properties:
batch_id:
description: The ID of the batch to cancel.
title: Batch Id
type: string
required:
- batch_id
title: CancelBatchRequest
type: object
DialogType:
description: Parameter type for dialog data with semantic output labels.
properties:
Expand Down
72 changes: 72 additions & 0 deletions docs/static/experimental-llama-stack-spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -688,6 +688,40 @@ components:
- data
title: ListBatchesResponse
description: Response containing a list of batch objects.
CreateBatchRequest:
properties:
input_file_id:
type: string
title: Input File Id
description: The ID of an uploaded file containing requests for the batch.
endpoint:
type: string
title: Endpoint
description: The endpoint to be used for all requests in the batch.
completion_window:
type: string
const: 24h
title: Completion Window
description: The time window within which the batch should be processed.
metadata:
anyOf:
- additionalProperties:
type: string
type: object
- type: 'null'
description: Optional metadata for the batch.
idempotency_key:
anyOf:
- type: string
- type: 'null'
description: Optional idempotency key. When provided, enables idempotent behavior.
type: object
required:
- input_file_id
- endpoint
- completion_window
title: CreateBatchRequest
description: Request model for creating a batch.
Batch:
properties:
id:
Expand Down Expand Up @@ -8532,6 +8566,44 @@ components:
- query
title: VectorStoreSearchRequest
type: object
ListBatchesRequest:
description: Request model for listing batches.
properties:
after:
anyOf:
- type: string
- type: 'null'
description: Optional cursor for pagination. Returns batches after this ID.
nullable: true
limit:
default: 20
description: Maximum number of batches to return. Defaults to 20.
title: Limit
type: integer
title: ListBatchesRequest
type: object
RetrieveBatchRequest:
description: Request model for retrieving a batch.
properties:
batch_id:
description: The ID of the batch to retrieve.
title: Batch Id
type: string
required:
- batch_id
title: RetrieveBatchRequest
type: object
CancelBatchRequest:
description: Request model for canceling a batch.
properties:
batch_id:
description: The ID of the batch to cancel.
title: Batch Id
type: string
required:
- batch_id
title: CancelBatchRequest
type: object
DialogType:
description: Parameter type for dialog data with semantic output labels.
properties:
Expand Down
Loading