[bug] OpenAI-to-Anthropic translation erases structured refusal text and emits empty text block

[Atharva-Kanherkar]

Symptom

When an OpenAI Chat backend returns a structured refusal (message.refusal populated with content: null), Switchyard's /v1/messages (Anthropic Messages) translation erases the refusal text and emits an empty text block [{"type": "text", "text": ""}] with stop_reason: "end_turn". The client caller receives a successful turn with empty content and has no indication that the upstream model refused the request.

Reproduction

1. Start switchyard-server pointing to an OpenAI-compatible backend configured with an openai_chat client.
2. Return a standard OpenAI Chat completion refusal payload:

{
  "id": "chatcmpl-test",
  "object": "chat.completion",
  "created": 0,
  "model": "captured-model",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": null,
        "refusal": "REFUSALPROBE cannot help"
      }
    }
  ],
  "usage": { "prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2 }
}

3. Issue an Anthropic Messages request:

curl -s -X POST http://localhost:14003/v1/messages \
  -H 'content-type: application/json' \
  -H 'x-api-key: test' \
  -d '{"model":"captured-model","max_tokens":64,"messages":[{"role":"user","content":"hi"}]}'

Expected vs. actual

• Expected: The refusal string is preserved and accessible to the Anthropic client (e.g., as a text block [{"type": "text", "text": "REFUSALPROBE cannot help"}] and stop_reason: "refusal" with stop_details).
• Actual: HTTP 200 with:

{
  "id": "chatcmpl-test",
  "type": "message",
  "role": "assistant",
  "model": "captured-model",
  "content": [
    {
      "type": "text",
      "text": ""
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "stop_details": null,
  "usage": { "input_tokens": 1, "output_tokens": 1 }
}

The refusal string "REFUSALPROBE cannot help" is erased completely, and an empty text block is invented.

Environment

• Switchyard version (or commit SHA): 4022b677 (and release 0.2.0 at 9523023)
• OS / arch: macOS arm64 / Linux x86_64
• Inbound format: Anthropic Messages (/v1/messages)
• Backend: OpenAI Chat Completions (/v1/chat/completions)

Controlled A/B evidence

Client endpoint	Upstream refusal payload	Client result	Outcome
Anthropic /v1/messages	{"content": null, "refusal": "..."}	content: [{"type": "text", "text": ""}], stop_reason: "end_turn"	Refusal erased, empty block invented (5/5)
OpenAI /v1/chat/completions	{"content": null, "refusal": "..."}	message.refusal: "...", content: null	Refusal preserved (5/5 control pass)

The same Switchyard process and identical upstream response preserve the refusal when requested via the /v1/chat/completions ingress, confirming the loss is isolated to the OpenAI-to-Anthropic translation path.

Root cause analysis

In crates/switchyard-translation/src/codecs/openai_chat/buffered.rs, decode_response:

1. Line 287 reads message.get("content").unwrap_or(&Value::Null). When content is null (standard for OpenAI Chat refusals), decode_openai_content evaluates Value::Null to vec![ContentBlock::Text { text: String::new() }].
2. message.get("refusal") is never inspected in decode_response. (The only "refusal" handling in openai_chat/buffered.rs is at line 513 inside decode_openai_content, which matches structured content array blocks like {"type": "refusal", "refusal": "..."}, not the Chat Completions message sibling field message.refusal).
3. If finish_reason is "stop", map_openai_finish_reason maps it to StopReason::EndTurn.
4. During Anthropic encoding (codecs/anthropic/buffered.rs), the normalized ContentBlock::Text { text: "" } is emitted as {"type": "text", "text": ""} and stop_reason is "end_turn". (Anthropic's block encoder at line 874 already supports ContentBlock::Refusal { text }, mapping it to a text block, but ContentBlock::Refusal is never created by the decoder).

Streaming in codecs/openai_chat/stream.rs exhibits a similar gap: lines 129-136 decode delta.get("content"), but delta.get("refusal") is not decoded.

Proposed solution & possible patches

1. Buffered decode (crates/switchyard-translation/src/codecs/openai_chat/buffered.rs)

In decode_response:

            if let Some(refusal) = message.get("refusal").and_then(Value::as_str) {
                // If content was null, drop the empty placeholder block and emit Refusal
                if content.len() == 1
                    && matches!(&content[0], ContentBlock::Text { text } if text.is_empty())
                {
                    content.clear();
                }
                content.push(ContentBlock::Refusal {
                    text: refusal.to_string(),
                });
            }

And adjust stop_reason:

            let raw_finish_reason = choice.get("finish_reason").and_then(Value::as_str);
            let stop_reason = if message.get("refusal").and_then(Value::as_str).is_some()
                && (raw_finish_reason == Some("stop") || raw_finish_reason.is_none())
            {
                StopReason::ContentFilter
            } else {
                map_openai_finish_reason(raw_finish_reason)
            };

Setting StopReason::ContentFilter allows Anthropic encoding to set stop_reason: "refusal" and populate Anthropic stop_details, while rendering the refusal explanation in the text content.

2. Stream decode (crates/switchyard-translation/src/codecs/openai_chat/stream.rs)

In decode_chunk:

            if let Some(text) = delta.get("refusal").and_then(Value::as_str)
                && !text.is_empty()
            {
                out.push(LlmResponseChunk::TextDelta {
                    index: 0,
                    text: text.to_string(),
                });
            }

Risk and impact

1. Safety and Policy Refusal Erasure: When an upstream model declines an unsafe or policy-violating request via structured refusal, the refusal explanation is swallowed.
2. False Success in Agent Workflows: Clients such as Claude Code or Anthropic SDK agent loops receive stop_reason: "end_turn" with an empty string. The agent framework interprets this as a successful completion rather than a refusal, potentially resulting in confusion, repeat retries, or execution of empty tool results.
3. Safety Evaluator Blindness: Benchmark, auditing, or guardrail tools inspecting Anthropic responses will fail to detect model refusals, reporting false negatives.
4. Invariant Violation: Dialect translation must preserve upstream semantic signals. Swapping an upstream refusal for an invented empty string violates response fidelity.

Additional context

• Related to PR fix(translation): report content filter stops as Anthropic refusal #370 (fix(translation): report content filter stops as Anthropic refusal), which added mapping for finish_reason: "content_filter", but did not address the message.refusal string or payload field on the message object.

[roma5087]

Can this be assigned to me?

[Atharva-Kanherkar]

I kinda made a pr for it earlier. you might push patches/review it. neither do I hold the authority to assign you.