feat: verification modes + evidence fields + transport combo rejection (hosted parity)

CHANGELOG.md

@@ -2,6 +2,20 @@
 
 All notable changes to cueapi-core will be documented here.
 
+## [Unreleased]
+
+### Added
+- **Verification modes** for cue outcomes. A new `verification: {mode: ...}` field on `CueCreate` / `CueUpdate` with five values: `none` (default), `require_external_id`, `require_result_url`, `require_artifacts`, `manual`. The outcome service computes `outcome_state` from (success, mode, evidence): missing required evidence lands in `verification_failed`, satisfied requirements land in `verified_success`, manual mode parks in `verification_pending`.
+- **Inline evidence on `POST /v1/executions/{id}/outcome`.** `OutcomeRequest` now accepts `external_id`, `result_url`, `result_ref`, `result_type`, `summary`, `artifacts` alongside the existing `success` / `result` / `error` / `metadata`. Fully backward compatible — the legacy shape still works. The separate `PATCH /v1/executions/{id}/evidence` endpoint remains for two-step flows.
+- **Migration 017** — `verification_mode` column on `cues` (String(50), nullable, CHECK-constrained enum). NULL and `none` are equivalent.
+
+### Changed
+- **`POST /v1/executions/{id}/verify`** now accepts `{valid: bool, reason: str?}`. `valid=true` (default, preserving legacy behavior) transitions to `verified_success`; `valid=false` transitions to `verification_failed` and records the reason onto `evidence_summary` (truncated to 500 chars). Accepted starting states expanded to include `reported_failure`.
+- `OutcomeResponse` now surfaces `outcome_state` in the response body.
+
+### Restricted
+- Worker-transport cues cannot currently combine with evidence-requiring verification modes (`require_external_id`, `require_result_url`, `require_artifacts`). Attempting to create or PATCH such a combination returns `400 unsupported_verification_for_transport`. `none` and `manual` are allowed for worker cues. This restriction will be lifted once cueapi-worker 0.3.0 (evidence reporting via `CUEAPI_OUTCOME_FILE`) is on PyPI.
+
 ## [0.1.2] - 2026-03-28
 
 ### Security

README.md

@@ -198,6 +198,56 @@ curl -X POST http://localhost:8000/v1/worker/heartbeat \
 
 The handlers array tells CueAPI which cue names this worker can process.
 
+### Verification modes
+
+Cues can require evidence on the outcome report. Configure a `verification` policy at create or update time:
+
+```bash
+curl -X POST http://localhost:8000/v1/cues \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -d '{"name": "nightly-report", "schedule": {"type": "recurring", "cron": "0 9 * * *"},
+       "callback": {"url": "https://your-handler.com"},
+       "verification": {"mode": "require_external_id"}}'
+```
+
+Five modes:
+
+| Mode | Behavior |
+|------|----------|
+| `none` (default) | Reported `success` is final — `reported_success` / `reported_failure`. |
+| `require_external_id` | Outcome must include `external_id`. Missing → `verification_failed`. Present → `verified_success`. |
+| `require_result_url` | Outcome must include `result_url`. |
+| `require_artifacts` | Outcome must include `artifacts` (non-empty). |
+| `manual` | Every successful outcome parks in `verification_pending` until someone calls `POST /v1/executions/{id}/verify`. |
+
+Report outcomes with evidence inline on the existing endpoint:
+
+```bash
+curl -X POST http://localhost:8000/v1/executions/EXEC_ID/outcome \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -d '{"success": true, "external_id": "stripe_ch_abc123",
+       "result_url": "https://dashboard.stripe.com/payments/ch_abc123",
+       "summary": "Charged customer 42"}'
+```
+
+Manually verify or reject a parked outcome:
+
+```bash
+# Approve
+curl -X POST http://localhost:8000/v1/executions/EXEC_ID/verify \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -d '{"valid": true}'
+
+# Reject (e.g. after audit)
+curl -X POST http://localhost:8000/v1/executions/EXEC_ID/verify \
+  -H "Authorization: Bearer YOUR_API_KEY" \
+  -d '{"valid": false, "reason": "invoice number does not match"}'
+```
+
+Backward-compat paths still work: `POST /outcome` with just `{success: true}` behaves identically to before, and `PATCH /v1/executions/{id}/evidence` remains available as a two-step alternative.
+
+> Worker-transport cues can currently use `none` or `manual` only. Evidence-requiring modes (`require_external_id`, `require_result_url`, `require_artifacts`) are rejected at create/update time with `400 unsupported_verification_for_transport`. This restriction will be lifted once cueapi-worker 0.3.0 ships to PyPI with evidence reporting via `CUEAPI_OUTCOME_FILE`.
+
 ## What CueAPI is not
 
 - Not a workflow orchestrator

alembic/versions/017_add_verification_mode.py

@@ -0,0 +1,41 @@
+"""Add verification_mode column to cues table.
+
+Outcome-verification policy per cue. Stored as a plain string rather
+than JSONB because the only structured field today is `mode`; keeping
+it a string keeps queries simple (`WHERE verification_mode = ...`) and
+lets Postgres enforce the enum via a CHECK constraint. If the policy
+gains fields later, widen to JSONB with a separate migration.
+
+NULL means "no verification" — equivalent to mode=none but avoids a
+row rewrite for the 100% of existing rows that have never configured
+verification. Outcome service treats NULL and 'none' identically.
+
+Revision ID: 017
+Revises: 016
+"""
+from alembic import op
+import sqlalchemy as sa
+
+revision = "017"
+down_revision = "016"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "cues",
+        sa.Column("verification_mode", sa.String(length=50), nullable=True),
+    )
+    op.create_check_constraint(
+        "valid_verification_mode",
+        "cues",
+        "verification_mode IS NULL OR verification_mode IN ("
+        "'none', 'require_external_id', 'require_result_url', "
+        "'require_artifacts', 'manual')",
+    )
+
+
+def downgrade() -> None:
+    op.drop_constraint("valid_verification_mode", "cues", type_="check")
+    op.drop_column("cues", "verification_mode")

app/models/cue.py

@@ -28,6 +28,8 @@ class Cue(Base):
     run_count = Column(Integer, nullable=False, default=0)
     fired_count = Column(Integer, nullable=False, default=0)
     on_failure = Column(JSONB, nullable=True, default={"email": True, "webhook": None, "pause": False})
+    # Outcome-verification policy. NULL == no verification (same as 'none').
+    verification_mode = Column(String(50), nullable=True, default=None)
     created_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
     updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now())
 
@@ -36,5 +38,11 @@ class Cue(Base):
         CheckConstraint("schedule_type IN ('once', 'recurring')", name="valid_schedule_type"),
         CheckConstraint("callback_method IN ('POST', 'GET', 'PUT', 'PATCH')", name="valid_callback_method"),
         CheckConstraint("callback_transport IN ('webhook', 'worker')", name="valid_callback_transport"),
+        CheckConstraint(
+            "verification_mode IS NULL OR verification_mode IN ("
+            "'none', 'require_external_id', 'require_result_url', "
+            "'require_artifacts', 'manual')",
+            name="valid_verification_mode",
+        ),
         UniqueConstraint("user_id", "name", name="unique_user_cue_name"),
     )

app/routers/executions.py

@@ -3,7 +3,8 @@
 from datetime import datetime, timedelta, timezone
 from typing import Optional
 
-from fastapi import APIRouter, Depends, Header, HTTPException, Query
+from fastapi import APIRouter, Body, Depends, Header, HTTPException, Query
+from pydantic import BaseModel
 from sqlalchemy import select, update
 from sqlalchemy.ext.asyncio import AsyncSession
 
@@ -331,29 +332,86 @@ async def replay_execution(
 # ── Verify ──
 
 
+class VerifyRequest(BaseModel):
+    """Body for ``POST /v1/executions/{id}/verify``.
+
+    Body is optional — a request with no body (or ``{}``) defaults to
+    ``valid=true`` so the previous always-success behavior remains the
+    default. ``valid=false`` is the new branch: it transitions to
+    ``verification_failed`` and optionally persists a human-readable
+    ``reason`` onto ``evidence_summary``.
+    """
+
+    valid: bool = True
+    reason: Optional[str] = None
+
+
 @router.post("/{execution_id}/verify")
 async def verify_execution(
     execution_id: str,
+    body: Optional[VerifyRequest] = Body(None),
     user: AuthenticatedUser = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    """Mark execution outcome as verified."""
+    """Mark execution outcome as verified or verification-failed.
+
+    Accepts ``{valid: bool, reason: str?}``. ``valid=true`` (default)
+    transitions to ``verified_success``; ``valid=false`` transitions to
+    ``verification_failed`` and records the reason on
+    ``evidence_summary`` (truncated to 500 chars). Accepted starting
+    states: ``reported_success``, ``reported_failure``,
+    ``verification_pending``.
+    """
     result = await db.execute(
         select(Execution).join(Cue, Execution.cue_id == Cue.id)
         .where(Execution.id == execution_id, Cue.user_id == user.id)
     )
     execution = result.scalar_one_or_none()
     if not execution:
-        raise HTTPException(status_code=404)
+        raise HTTPException(status_code=404, detail={"error": {"code": "execution_not_found", "message": "Execution not found", "status": 404}})
 
-    if execution.outcome_state not in {"reported_success", "verification_pending"}:
-        raise HTTPException(status_code=409, detail={"error": {"code": "invalid_state", "message": f"Cannot verify from state '{execution.outcome_state}'"}})
+    if execution.outcome_state not in {
+        "reported_success",
+        "reported_failure",
+        "verification_pending",
+    }:
+        raise HTTPException(
+            status_code=409,
+            detail={
+                "error": {
+                    "code": "invalid_state",
+                    "message": f"Cannot verify from state '{execution.outcome_state}'",
+                    "status": 409,
+                }
+            },
+        )
 
-    execution.outcome_state = "verified_success"
-    execution.evidence_validation_state = "valid"
-    execution.updated_at = datetime.now(timezone.utc)
+    payload = body or VerifyRequest()
+    now = datetime.now(timezone.utc)
+    if payload.valid:
+        execution.outcome_state = "verified_success"
+        execution.evidence_validation_state = "valid"
+    else:
+        execution.outcome_state = "verification_failed"
+        execution.evidence_validation_state = "invalid"
+        if payload.reason:
+            # Persist reason alongside any existing summary; truncate
+            # to the column cap. We prepend so operators who set a
+            # summary at outcome-report time still see it.
+            truncated = payload.reason[:500]
+            if execution.evidence_summary:
+                combined = f"{execution.evidence_summary} | verification rejected: {truncated}"
+                execution.evidence_summary = combined[:500]
+            else:
+                execution.evidence_summary = truncated
+    execution.updated_at = now
     await db.commit()
-    return {"execution_id": str(execution_id), "outcome_state": "verified_success"}
+    return {
+        "execution_id": str(execution_id),
+        "outcome_state": execution.outcome_state,
+        "valid": payload.valid,
+        "reason": payload.reason,
+    }
 
 
 # ── Verification pending ──

app/schemas/cue.py

@@ -1,13 +1,42 @@
 from __future__ import annotations
 
 from datetime import datetime
+from enum import Enum
 from typing import Dict, List, Optional
 
 from pydantic import BaseModel, ConfigDict, Field, HttpUrl, field_validator, model_validator
 
 from app.schemas.execution import ExecutionResponse
 
 
+class VerificationMode(str, Enum):
+    """Outcome-verification policy for a cue.
+
+    - ``none``: the reported ``success`` bool is final. Default.
+    - ``require_external_id``/``result_url``/``artifacts``: evidence
+      field must be present on the outcome report; if missing, the
+      execution is marked ``verification_failed``. If present and
+      ``success=True``, the execution is marked ``verified_success``.
+    - ``manual``: every successful outcome sits in
+      ``verification_pending`` until someone calls
+      ``POST /v1/executions/{id}/verify``.
+    """
+
+    none = "none"
+    require_external_id = "require_external_id"
+    require_result_url = "require_result_url"
+    require_artifacts = "require_artifacts"
+    manual = "manual"
+
+
+class VerificationPolicy(BaseModel):
+    """Outcome-verification policy. Only ``mode`` today; kept as a
+    sub-object so future fields (e.g. ``auto_verify_after``) can be
+    added without breaking the API shape."""
+
+    mode: VerificationMode = Field(default=VerificationMode.none)
+
+
 class ScheduleCreate(BaseModel):
     type: str  # "once" | "recurring"
     cron: Optional[str] = None
@@ -43,6 +72,7 @@ class CueCreate(BaseModel):
     payload: Optional[dict] = Field(default={})
     retry: Optional[RetryConfig] = Field(default_factory=RetryConfig)
     on_failure: Optional[OnFailureConfig] = Field(default_factory=OnFailureConfig)
+    verification: Optional[VerificationPolicy] = None
 
     @model_validator(mode="after")
     def validate_transport(self) -> "CueCreate":
@@ -73,6 +103,7 @@ class CueUpdate(BaseModel):
     payload: Optional[dict] = None
     retry: Optional[RetryConfig] = None
     on_failure: Optional[OnFailureConfig] = None
+    verification: Optional[VerificationPolicy] = None
 
     @field_validator("status")
     @classmethod
@@ -97,6 +128,7 @@ class CueResponse(BaseModel):
     run_count: int
     fired_count: int = 0
     on_failure: Optional[dict] = None
+    verification: Optional[dict] = None
     warning: Optional[str] = None
     created_at: datetime
     updated_at: datetime

app/schemas/outcome.py

@@ -1,18 +1,35 @@
 from __future__ import annotations
 
-from typing import Any, Dict, Optional
+from typing import Any, Dict, List, Optional
 
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, Field, HttpUrl
 
 
 class OutcomeRequest(BaseModel):
+    """Outcome report. Evidence fields are optional and additive — a
+    caller that only sends {success, result, error, metadata} gets the
+    identical behavior it always did. Evidence fields feed the
+    verification-modes policy configured on the cue (see
+    ``VerificationMode``)."""
+
     success: bool
     result: Optional[str] = Field(None, max_length=2000)
     error: Optional[str] = Field(None, max_length=2000)
     metadata: Optional[Dict[str, Any]] = None
+    # Evidence fields — recorded on the Execution's ``evidence_*``
+    # columns. Any missing evidence required by the cue's verification
+    # mode causes the outcome to land in ``verification_failed`` rather
+    # than ``reported_success``.
+    external_id: Optional[str] = Field(None, max_length=500)
+    result_url: Optional[HttpUrl] = None
+    result_ref: Optional[str] = Field(None, max_length=500)
+    result_type: Optional[str] = Field(None, max_length=100)
+    summary: Optional[str] = Field(None, max_length=500)
+    artifacts: Optional[List[Any]] = None
 
 
 class OutcomeResponse(BaseModel):
     execution_id: str
     outcome_recorded: bool
+    outcome_state: Optional[str] = None
     reason: Optional[str] = None