Skip to content

Commit

Permalink
Update fastapi.md
Browse files Browse the repository at this point in the history
  • Loading branch information
waseemhnyc committed Dec 6, 2023
1 parent e5979bd commit c0fb798
Showing 1 changed file with 20 additions and 1 deletion.
21 changes: 20 additions & 1 deletion docs/concepts/fastapi.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,32 @@ def endpoint_function(data: UserData) -> UserDetail:
`FastAPI` supports streaming responses, which is useful for returning large amounts of data. This feature is particularly useful when working with large language models (LLMs) that generate a large amount of data.

```python hl_lines="6-7"
import instructor

from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from openai import OpenAI
from typing import Iterable

# Enables response_model
client = instructor.patch(OpenAI())
app = FastAPI()

class UserData(BaseModel):
# This can be the model for the input data
query: str

class UserDetail(BaseModel):
name: str
age: int


# Route to handle SSE events and return users
@app.post("/extract", response_class=StreamingResponse)
async def extract(data: UserData):
users = await client.chat.completions.create(

users = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=Iterable[UserDetail],
stream=True,
Expand Down

0 comments on commit c0fb798

Please sign in to comment.