Getting Structured LLM Output 📊🤖

• Code files from the deeplearning.ai short course on Structured LLM Output
• Certificate
• Handwritten Notes

Why Structured LLM Outputs? 🧩

Structured outputs transform raw LLM responses into predictable, machine-readable formats that:

• Ensure consistency in response format and structure
• Simplify integration with downstream systems and databases
• Reduce hallucinations by constraining output scope and format
• Improve reliability for production applications
• Enable validation of output against schema requirements
• Facilitate parsing without complex regex or post-processing

Types of Structured Output Approaches 🛠️

1. Schema-Based (OpenAI)

Uses JSON schema to define the structure directly in the API call, enforced by the model itself.

client.beta.chat.completions.parse(
    response_format=MySchema
)

2. Retry-Based (Instructor)

Iteratively improves outputs by validating against schemas and retrying when validation fails.

instructor_client.chat.completions.create(
    response_model=MySchema,
    max_retries=3
)

3. Grammar-Constrained (Outlines)

Uses parsing expression grammars to constrain token generation at inference time.

outlines.generate.json(model, MySchema)
# or
outlines.generate.regex(model, pattern)

No.	Concepts	NBSanity	GitHub
1	• Defining output schemas with Pydantic • Using OpenAI's parse() for structured output • Sentiment analysis of social media mentions • Converting structured data to pandas DataFrames
2	• Using Instructor for structured outputs • Retry-based output generation • Error handling with max_retries • Custom regex validation in Pydantic models • Tracking token usage with retries
3	• Structured generation with Outlines • JSON output generation with constraints • Visualizing token probabilities
4	• Structured generation beyond JSON • Classification with choice() • Regex-based output generation • HTML, CSV and pattern-based outputs • Simplifying regex with DSL • Multi-modal structured outputs (vision)

Resources 📚

• OpenAI Structured Outputs Guide
• Instructor Documentation
• Outlines Documentation