LLMFlow

Orchestration engine & UI for your customized LLM flow.

Installation

go install github.com/go-aie/llmflow/cmd/llmflow@latest

Quick Start

Run the basic flow:

OPENAI_API_KEY=<YOUR_API_KEY> llmflow run examples/basic.flow.yaml

Start LLMFlow

API

Start LLMFlow:

OPENAI_API_KEY=<YOUR_API_KEY> llmflow serve

Add the basic flow (in YAML format):

curl -XPUT -H 'Content-Type: application/yaml' 'http://127.0.0.1:8888/api/flows/basic' --data-binary @examples/basic.flow.yaml

Or add the basic flow (in JSON format):

curl -XPUT -H 'Content-Type: application/json' 'http://127.0.0.1:8888/api/flows/basic' -d @examples/basic.flow.json

Execute the basic flow:

curl -XPOST -H 'Content-Type: application/json' 'http://127.0.0.1:8888/api/flows/basic:run' -d '{"query":"colorful socks"}'

NOTE:

• Setting environment variables is not mandatory when running LLMFlow in this example.
• As an alternative, you can change api_key to your API key in basic.flow.yaml (see UI), and then run llmflow instead.

UI

Open LLMFlow UI through your browser: http://127.0.0.1:8888.

Then click the Open button and select basic.flow.yaml to view it in the UI.

Tasks & Flows

In addition to Orchestrator's built-in tasks, LLMFlow defines the following tasks and flows:

• Tasks

  • JSONLinesLoader
  • Splitter
  • Template
  • TextLoader
  • VectorStore_Delete
  • VectorStore_Query
  • VectorStore_Upsert

• Flows

  • Embedding_OpenAI
  • Embedding_Azure_OpenAI
  • LLM_OpenAI
  • LLM_Azure_OpenAI
  • LLM_Ollama
  • xHTTP
  • Bing_Search

Examples

• Basic
• Document Question Answering
  • Feed
  • Query
• FAQ-based Question Answering
  • Feed
  • Query
• Table Question Answering (requires TableQA)
• Table & Document Question Answering (requires TableQA)
  • Feed
  • Query
• Tool Call
• Perplexity

Documentation

Check out the documentation.

License

Apache License 2.0