Skip to content

feat: add OpenAI annotator support#42

Merged
tyzbit merged 2 commits into
tyzbit:mainfrom
joryirving:openai-annotator
May 20, 2026
Merged

feat: add OpenAI annotator support#42
tyzbit merged 2 commits into
tyzbit:mainfrom
joryirving:openai-annotator

Conversation

@joryirving
Copy link
Copy Markdown
Contributor

@joryirving joryirving commented May 20, 2026

What

Add OpenAI annotator support to the Annotate step, matching the existing OpenAI filter support.

Closes #41

Why

Users running local inference servers (llama.cpp, text-generation-webui, etc.) can now use both filtering and annotation without needing a separate Ollama instance. This is especially useful for CPU-only deployments where Ollama's overhead is undesirable.

Changes

  • New: annotator_openai.go — OpenAI annotator implementation (166 lines)
    • Uses openai-go client with configurable base URL
    • Regex-based JSON extraction from chat responses (compatible with any OpenAI-compatible server)
    • Same response format as Ollama annotator: model_feedback, processed_number, processed_text, question_answer
    • Supports API key, model selection, timeout, system prompt override
    • Default timeout 60s for slower CPU models
  • Modified: config.go — added OpenAI OpenAIAnnotator field to AnnotateStep struct
  • Modified: annotators.go — registered OpenAI annotator in the annotators slice

Example config

Steps:
- Filter:
    OpenAI:
      Model: qwen3.5-9b
      URL: http://llama.llm:8080/v1
      UserPrompt: >
        Should this message be forwarded? Look for human-written content...
- Annotate:
    OpenAI:
      Model: qwen3.5-9b
      URL: http://llama.llm:8080/v1
      UserPrompt: >
        Rate the emotional content of this message from pilots and ATC
        between 1-100 for any emotions such as anger, frustration, etc.

Add OpenAI-compatible API support to the Annotate step, matching
the existing OpenAI filter support. This allows users running
local inference servers (llama.cpp, text-generation-webui, etc.)
to use both filtering and annotation without a separate Ollama
instance.

The annotator uses regex-based JSON extraction from chat responses
(consistent with the existing OpenAI filter) rather than relying
on response_format JSON schemas, making it compatible with any
OpenAI-compatible server.
Copy link
Copy Markdown
Owner

@tyzbit tyzbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor question. Looks great to me though, thank you!

Comment thread annotator_openai.go Outdated
@joryirving joryirving requested a review from tyzbit May 20, 2026 16:16
@tyzbit tyzbit merged commit 68877f1 into tyzbit:main May 20, 2026
@joryirving joryirving deleted the openai-annotator branch May 20, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add OpenAI support for Annotate step

2 participants