dspy-cli

dspy-cli is a tool for creating, developing, testing, and deploying DSPy programs as HTTP APIs. dspy-cli auto-generates endpoints, OpenAPI specs, and Docker configs.

With dspy-cli, you get:

• A standard project layout for DSPy programs.
• A FastAPI server with auto-generated endpoints.
• OpenAPI specs and an optional MCP server.
• Dockerfile and logging out of the box.

Quick Start

Installing & Creating a New Project

To install dspy-cli, we recommend using uv.

uv tool install dspy-cli

You can also use pip install dspy-cli, but the example below assumes uv.

Running dspy-cli new creates a new project, which can have many programs, each of which performs a specific task.

To illustrate, let's build a project that manages AI-powered functions we might want to call from a content management system we use to draft, publish, and manage blog posts. We'll call it "cms-kit".

dspy-cli new cms-kit

This command launches an interactive menu that walks you through setting up your first program and connecting to an inference provider.

Here are our answers:

Would you like to specify your first program? [Y/n]: Y
What is the name of your first DSPy program? [my_program]: summarizer

The first program we'll build will write short summaries of our blog posts. We'll call it summarizer.

Continuing:

Choose a module type:
  1. Predict - Basic prediction module (default)
  2. ChainOfThought (CoT) - Step-by-step reasoning with chain of thought
  3. ProgramOfThought (PoT) - Generates and executes code for reasoning
  4. ReAct - Reasoning and acting with tools
  5. MultiChainComparison - Compare multiple reasoning paths
  6. Refine - Iterative refinement of outputs
Enter number or name [1]: 1
Enter your signature or type '?' for guided input:
  Examples: 'question -> answer', 'post:str -> tags:list[str], category:str'
Signature [question:str -> answer:str]: blog_post -> summary

Here we're using a basic Predict module and specifying our signature as "blog_post -> summary".

Lastly, we'll connect to the model we want to use:

Enter your model (LiteLLM format):
  Examples: 'anthropic/claude-sonnet-4-5', 'openai/gpt-4o', 'ollama/llama2'
Model [openai/gpt-5-mini]: openai/gpt-5-mini
Enter your OpenAI API key:
  (This will be stored in .env as OPENAI_API_KEY)
  Press Enter to skip and set it manually later
OPENAI_API_KEY: your_key_here

DSPy uses LiteLLM to connect to language models, so we're using a LiteLLM style string to call GPT-5-mini. Paste in your OpenAI key, and you're good to go. dspy-cli will attempt to detect any API key variables in your local environment and will pre-populate this field if a candidate is found.

dspy-cli will now create your project structure and define your first program. Let's cd into the folder cms-kit, activate our environment, and serve our project.

cd cms-kit
uv sync
source .venv/bin/activate
dspy-cli serve

dspy-cli will detect the module we've defined, define an endpoint for it, then stand up an HTTP server to call, at http://localhost:8000 (by default). Visiting that URL will let you submit a form to call your program:

Or, you can call the API endpoint directly, like so:

curl -X POST http://0.0.0.0:8000/SummarizerPredict \
  -H "Content-Type: application/json" \
  -d '{
  "blog_post": "[AN EXAMPLE BLOG POST]"
}'

Here, /SummarizerPredict is the auto-generated route name corresponding to the DSPy module created for our summarizer program.

Creating Another Program

In addition to summarizing blog posts, we can imagine several other LLM-powered functions we could perform in our CMS: tagging, image description writing, drafting social media posts, etc.

To add a new program to cms-kit, we can run the generate scaffold command:

dspy-cli generate scaffold tagger -s "blog_post -> tags:list[str]"

We name our program tagger and use the -s or --signature flag to pass in a signature detailing passing in a blog post and getting back a list of tags, which we specify as list[str].

generate scaffold creates our program by creating a signature and module file in src/cms_kit/signatures and src/cms_kit/modules, respectively.

If we run dspy-cli serve, the new module will be discovered and hosted in the web UI and as a new API route.

Exploring Our Project

Running dspy-cli new sets up your project directory. Let's walk through a few key items created:

• src/cms_kit/signatures: Class-based DSPy signatures are created and housed here.
• src/cms_kit/modules: DSPy modules are stored here. This folder is where dspy-cli discovers available programs when running serve.
• src/cms_kit/utils: While you can add arbitrary code to your module and signature files, utils is a handy place to stash additional logic and tool definitions. Just create, code, and import from your signatures and modules.
• logs: When your programs are called (via the web UI or API calls), usage is logged to a program-specific JSONL file, in the logs folder.
• dspy.config.yaml: This config file sets a few parameters, but is mainly where inference providers and models are defined, both globally and (if you want) on a per-program basis.

Speaking of model definitions...

Connecting to Models

In dspy.config.yaml you're able to define language models you intend to call. We define models in a YAML format, with similar parameters to the LiteLLM integration DSPy uses.

Here's what our model registry looks like in our cms-kit project:

models:
  # The default model to use if no per-program override is specified
  default: openai:gpt-5-mini

  # Model registry - define all available models here
  registry:
    openai:gpt-5-mini:
      model: openai/gpt-5-mini
      model_type: chat
      max_tokens: 16000
      temperature: 1.0
      env: OPENAI_API_KEY

In the registry list we define models, using the LiteLLM convention, like: "openai/gpt-5-mini".

To add more models, extend the registry:

models:
  default: openai:gpt-5-mini

  registry:
    openai:gpt-5-mini:
      model: openai/gpt-5-mini
      model_type: chat
      max_tokens: 16000
      temperature: 1.0
      env: OPENAI_API_KEY

    anthropic:sonnet-4.5:
      model: anthropic/claude-sonnet-4-5
      env: ANTHROPIC_API_KEY
      max_tokens: 8192
      temperature: 0.7
      model_type: chat

    # A local model hosted with LM Studio
    qwen:qwen-3-4b:
      model: openai/qwen/qwen3-4b
      api_base: http://127.0.0.1:1234/v1
      api_key: placeholder
      max_tokens: 4096
      temperature: 1.0
      model_type: chat

The default key in the models list specifies the default model your programs will call. If you'd like, you can assign different programs to different models, like so:

program_models:
   TaggerPredict: qwen:qwen-3-4b
   SummarizerPredict: anthropic:sonnet-4.5

The env variable for each model refers to an API key defined in an .env file located at the root of your project directory.

Learning More

We've built out a lot of quality of life features in dspy-cli, including:

• Auto-discovery of modules: Create a module in the modules folder and dspy-cli will detect the program and infer its parameters.
• Type validation: Typed parameters and return values defined in a module's forward method are validated during API calls.
• Hot-reloading: Tweaking signature or module definitions will cause dspy-cli to reload the server, updating the programs.
• OpenAPI spec generation: With each run of serve, dspy-cli creates an OpenAPI JSON definition of your API, which is accessible at /openapi.json.
• MCP tool support: Pass in --mcp while calling serve to stand up an MCP server for your program.
• Docker configuration: Running new creates a Dockerfile, which can be used to quickly stand up your program in Docker.

Check out the full docs to learn more.

License

MIT