Skip to content

Pipeline Plan 242

Seth Ford edited this page Mar 10, 2026 · 5 revisions

I now have a complete picture. Here's the implementation plan:


Implementation Plan: Fix Misleading "jq not available" Warning (#242)

Root Cause Analysis

Root Cause Hypothesis (ranked by likelihood)

  1. Missing JSON object handler (95% confidence) — Case 2 only checks first_char == "[" (array). When Claude outputs {"type":"result",...} (object), the condition is false, so jq is never tried. Control falls through to Case 3, which misleadingly blames jq availability.

  2. Incorrect condition ordering — Case 3's guard first_char == "[" || first_char == "{" catches objects that Case 2 should have already handled.

  3. (Not applicable) — jq is actually missing. Ruled out: the command -v jq check in Case 2 is never reached for objects, so jq availability is irrelevant.

Evidence Gathered

  • File: scripts/sw-loop.sh:561-608_extract_text_from_json() function
  • Line 579: if [[ "$first_char" == "[" ]] && command -v jq — only handles arrays
  • Line 599-600: Falls through for { objects, prints misleading "jq not available"
  • Existing tests: Tests 17-19 and 21 in sw-loop-test.sh cover empty files, valid JSON arrays, plain text, nested arrays, and binary — but no test for JSON object input

Files to Modify

File Action
scripts/sw-loop.sh Extend Case 2 to handle JSON objects ({...}) alongside arrays ([...])
scripts/sw-loop-test.sh Add test for JSON object extraction + fix Case 3 warning message test

Implementation Steps

Step 1: Extend Case 2 in _extract_text_from_json to handle JSON objects

In scripts/sw-loop.sh lines 578-603, restructure the JSON parsing to:

  1. Check if first_char is [ or { AND jq is available (single combined guard)
  2. Inside the block, branch on array vs object for the jq expression:
    • Array: jq -r '.[-1].result // empty' (existing)
    • Object: jq -r '.result // empty' (new)
  3. Share the rest of the extraction logic (.content fallback, placeholder)
# Case 2: Valid JSON (array or object) — extract with jq
if [[ "$first_char" == "[" || "$first_char" == "{" ]] && command -v jq >/dev/null 2>&1; then
    local extracted
    if [[ "$first_char" == "[" ]]; then
        extracted=$(jq -r '.[-1].result // empty' "$json_file" 2>/dev/null) || true
    else
        extracted=$(jq -r '.result // empty' "$json_file" 2>/dev/null) || true
    fi
    if [[ -n "$extracted" ]]; then
        echo "$extracted" > "$log_file"
        return 0
    fi
    # jq succeeded but result was null/empty — try .content or raw text
    if [[ "$first_char" == "[" ]]; then
        extracted=$(jq -r '.[].content // empty' "$json_file" 2>/dev/null | head -500) || true
    else
        extracted=$(jq -r '.content // empty' "$json_file" 2>/dev/null | head -500) || true
    fi
    if [[ -n "$extracted" ]]; then
        echo "$extracted" > "$log_file"
        return 0
    fi
    # JSON parsed but no text found
    warn "JSON output has no .result field — check $json_file"
    echo "(no text result in JSON output)" > "$log_file"
    return 0
fi

Step 2: Fix Case 3 warning message

Case 3 now only triggers when jq is genuinely unavailable (since Case 2 handles both [ and { when jq exists). The message is now accurate, but add a clarifying distinction:

# Case 3: Looks like JSON but no jq — can't parse, use raw
if [[ "$first_char" == "[" || "$first_char" == "{" ]]; then
    warn "JSON output but jq not available — using raw output"
    cp "$json_file" "$log_file"
    return 0
fi

This message is now correct because it's only reached when command -v jq actually fails.

Step 3: Add test for JSON object input in sw-loop-test.sh

Add a new test case after the existing Test 19 (plain text passthrough) block that verifies:

  • JSON object {"type":"result","result":"Object extraction works",...} → extracts "Object extraction works"
  • No "jq not available" warning is emitted

Task Checklist

  • Task 1: Modify Case 2 guard in _extract_text_from_json to accept both [ and { first chars
  • Task 2: Add object-specific jq expressions (jq -r '.result // empty' and jq -r '.content // empty')
  • Task 3: Verify Case 3 warning is now only reachable when jq is genuinely missing (no code change needed — the logic is now correct by construction)
  • Task 4: Add test in sw-loop-test.sh for JSON object extraction (.result field)
  • Task 5: Add test for JSON object with .content field (fallback path)
  • Task 6: Add test verifying no false "jq not available" warning for JSON objects
  • Task 7: Run sw-loop-test.sh to verify all existing + new tests pass

Testing Approach

  1. Unit test (JSON object with .result): Feed {"type":"result","result":"Hello from object","subtype":"success"} → verify output contains "Hello from object"
  2. Unit test (JSON object with .content): Feed {"content":"Fallback content"} → verify output contains "Fallback content"
  3. Warning suppression test: Run extraction on a JSON object and capture stderr → verify "jq not available" does NOT appear
  4. Regression: Run existing tests 15-21 unchanged → all pass
  5. Run full test suite: ./scripts/sw-loop-test.sh

Definition of Done

  • _extract_text_from_json correctly extracts .result from both [...] and {...} JSON
  • No misleading "jq not available" warning when jq IS available
  • Case 3 warning only fires when jq is genuinely missing
  • New tests cover JSON object extraction (.result and .content fallback)
  • All existing sw-loop-test.sh tests continue to pass
  • No Bash 3.2 compatibility violations in the change

Fix Strategy

This is Option A from the issue description (extend Case 2), which is the correct approach because:

  • It fixes the root cause (jq never tried for objects), not just the symptom (wrong message)
  • It makes JSON object output from Claude actually work, extracting clean text instead of passing raw JSON through
  • The change is minimal — same jq patterns, just branching on [ vs {

Clone this wiki locally