Skip to content

feat(runtimes): http adapter — generic POST for Ollama, custom servers - #75

Merged
rlagowski merged 2 commits into
developfrom
issue/54-http-adapter
Apr 27, 2026
Merged

feat(runtimes): http adapter — generic POST for Ollama, custom servers#75
rlagowski merged 2 commits into
developfrom
issue/54-http-adapter

Conversation

@rafeekpro

Copy link
Copy Markdown
Collaborator

Summary

Replaces the F0 `HttpAdapter` stub with a working implementation. Pipelines can now hit any JSON-speaking service: Ollama, llama.cpp servers, internal LLM gateways, Anthropic/OpenAI-compatible proxies that don't fit the `api-call` (SDK) shape, custom team APIs.

Implementation

  • Sandboxed Jinja2 renders `request_template` recursively: strings get templated, numbers/booleans/lists pass through. Scope = `{prompt_xml, runtime_config}`.
  • jsonpath-ng extracts response fields. `response_extractor` is a `{string: jsonpath}` dict; matches lift into `RuntimeResult.{output, tokens_used, cost_usd}`. Non-string output coerces via `json.dumps`.
  • Auth via env vars (never persisted): bearer / header / basic / none. `auth.env` names the var; engine reads at request time.
  • `httpx.AsyncClient` with `task.timeout_ms`; cancellation via context manager kills the connection cleanly.
  • Static extra headers via `runtime_config.headers`.

Validation (strict, fail fast)

  • `url` required + must be http:// or https://
  • `method` ∈ {POST, PUT}
  • `request_template` required (dict/list/string)
  • `response_extractor` required, must include `"output"`, entries validated as JSONPath at validate time
  • `auth.type` ∈ {none, bearer, header, basic}; per-type required fields enforced

Failure paths

  • Missing/invalid config → 422-style `RuntimeResult.errors` (no request fired)
  • Missing auth env var → descriptive error referencing the var name
  • Network error / connection refused → HTTP request failed
  • Timeout → `timed_out=true` in structured
  • 4xx/5xx → preview of `response.text` in `errors[0]`
  • Non-JSON response → graceful failure
  • JSONPath miss → `output` coerces to `""`

Test plan

  • `tests/smoke/test_http_adapter.py` — 25 tests covering: validation, mocked POST, Jinja rendering (object + string forms), all 3 auth modes, static headers merge, nested-response JSONPath, non-string output coercion, all error paths
  • `uv run pytest tests/smoke/` — 255/255 pass
  • `uv run ruff check` + `ruff format --check apps packages` clean
  • `uv run mypy apps packages tests` clean (with new `[[tool.mypy.overrides]]` for `jsonpath_ng.*` — no py.typed upstream)
  • `pnpm tsc --noEmit` clean

Schema + deps

  • `runtime-config-schemas.ts`: http fields are now `required` for `url` + `request_template` + `response_extractor`. Header comment moves http to the implemented tier (5/7 runtimes done).
  • New deps in `dap-runtimes`: `jinja2>=3.1.4`, `jsonpath-ng>=1.6.1`.

Closes #54

🤖 Generated with Claude Code

Replaces the F0 HttpAdapter stub with a working implementation. Pipelines
can now hit any JSON-speaking service: Ollama, llama.cpp servers,
internal LLM gateways, Anthropic/OpenAI-compatible proxies that don't
fit the api-call (SDK) shape, custom team APIs.

## Implementation

- Sandboxed Jinja2 (jinja2 + jinja2.sandbox.SandboxedEnvironment) renders
  request_template recursively: strings get templated, numbers/booleans/
  lists pass through. Scope = {prompt_xml, runtime_config}.
- jsonpath-ng extracts response fields. response_extractor is a
  {string: jsonpath} dict; matches lift into RuntimeResult.{output,
  tokens_used, cost_usd}. Non-string output coerces via json.dumps.
- Auth via env vars (never persisted): bearer / header / basic / none.
  auth.env names the var; engine reads at request time.
- httpx.AsyncClient with task.timeout_ms; cancellation via context
  manager kills the connection cleanly.
- Static extra headers via runtime_config.headers.

## Validation

Strict — fails fast with descriptive messages:
- url required + must be http:// or https://
- method must be POST or PUT
- request_template required (dict/list/string)
- response_extractor required, must include "output", entries validated
  as JSONPath at validate time (not just at extract time)
- auth.type ∈ {none, bearer, header, basic}; per-type required fields

## Failure paths

- Missing/invalid config → 422-style RuntimeResult.errors (no request)
- Missing auth env var → descriptive error referencing the var name
- Network error / connection refused → HTTP request failed
- Timeout → timed_out=true in structured
- 4xx/5xx → preview of response.text in errors[0]
- Non-JSON response → graceful failure (suggests using a different runtime)
- JSONPath miss → output coerces to ""

## Tests

`tests/smoke/test_http_adapter.py` — 25 tests covering: validation,
mocked successful POST, request_template Jinja rendering (object +
string forms), all 3 auth modes, static headers merge, nested-response
JSONPath, non-string output coercion, all error paths above. Mocks
httpx.AsyncClient with async-context-manager support.

## Schema + deps

- `runtime-config-schemas.ts`: http fields are now `required` for url
  + request_template + response_extractor. Header comment moves http
  to the implemented tier (5/7 runtimes done).
- New deps in dap-runtimes: `jinja2>=3.1.4`, `jsonpath-ng>=1.6.1`.
- mypy override for jsonpath_ng.* (no py.typed marker upstream).

255/255 smoke tests pass; ruff + mypy + format clean.

Closes #54

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 27, 2026 14:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the previously-stubbed http runtime adapter so pipelines can call arbitrary JSON-speaking HTTP services (e.g., Ollama/llama.cpp/custom gateways), and wires the change through dependency management, UI schema validation, and documentation.

Changes:

  • Add a working HttpAdapter with Jinja2 request templating, env-var-based auth, and JSONPath response extraction.
  • Add comprehensive smoke tests for validation, auth modes, request/response handling, and error paths.
  • Update deps + typing config (jinja2/jsonpath-ng, mypy override), and tighten dashboard runtime-config schema/docs for http.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
packages/runtimes/src/dap_runtimes/adapters/http.py Replaces stub with full async HTTP adapter implementation (templating, auth, extraction, errors).
tests/smoke/test_http_adapter.py Adds smoke tests covering success paths, validation, auth, headers, JSONPath, and failure modes.
packages/runtimes/pyproject.toml Adds jinja2 and jsonpath-ng runtime dependencies.
uv.lock Locks new Python dependencies for the workspace.
pyproject.toml Adds mypy override for jsonpath_ng.* due to missing py.typed.
apps/dashboard/src/components/agents/runtime-config-schemas.ts Marks key http config fields as required and updates schema descriptions now that adapter is implemented.
packages/runtimes/README.md Updates runtime implementation status and documents the http runtime_config shape.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/runtimes/src/dap_runtimes/adapters/http.py Outdated
Comment thread packages/runtimes/src/dap_runtimes/adapters/http.py
Comment thread packages/runtimes/src/dap_runtimes/adapters/http.py Outdated
Comment thread packages/runtimes/src/dap_runtimes/adapters/http.py
Comment thread packages/runtimes/src/dap_runtimes/adapters/http.py Outdated
rlagowski
rlagowski previously approved these changes Apr 27, 2026
Five trafne komentarze z PR #75:

1. **`_build_headers` raises on non-string entries.** Previously
   silently dropped them; misconfigured headers were invisible. Now
   raises ValueError with the offending types in the message.

2. **Full payload no longer persisted in structured.** The unconditional
   `payload: ...` would land in NodeExecutionLog.output_json — bloats
   the DB and creates a data-retention surface for sensitive responses.
   Now `structured` carries `extracted` only (the user's named slices).
   Users who want raw debugging info can add `"_raw": "$"` to their
   response_extractor — explicit opt-in.

3. **`_apply_extractors` takes the first match without materialising
   the full list.** Lazy iteration with `break` on first hit; broad
   JSONPaths over large payloads are no longer linear in match count.

4. **Jinja2 uses StrictUndefined.** Previously a typo'd template
   variable rendered as empty string and silently shipped a malformed
   request. Now fails fast with a descriptive UndefinedError, matching
   prompt-dsl's existing behaviour. Aligns with the docstring claim
   of "same defence-in-depth as prompt-dsl".

5. **`headers` validated as dict[str, str] at config time.** Previously
   only checked outer dict-ness; non-string entries got silently
   dropped at request time. Now per-entry type check returns a 422-
   style error before the request is even sent.

Tests added: headers-must-be-dict-of-strings, strict-undefined-fails,
payload-not-persisted, raw-via-extractor (`$` JSONPath). 259/259
smoke pass; ruff + mypy + format clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rlagowski
rlagowski merged commit df97b4f into develop Apr 27, 2026
2 checks passed
@rlagowski
rlagowski deleted the issue/54-http-adapter branch May 10, 2026 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Runtime: http adapter (generic POST for Ollama, custom servers)

3 participants