fix(ai-suggestions): real-qsv schema + CLI flag corrections (PR #301/#302 follow-up)

[jqnatividad]

Summary

End-to-end verification against LM Studio (gemma-4-e4b on host.docker.internal:1234) surfaced two real bugs in the AI suggestions feature that landed in #301 + #302. The unit suite passed because subprocess.run was mocked with a synthetic JSON shape I invented — nothing exercised qsv's actual CLI surface or output schema.

Bug 1: --json flag doesn't exist in qsv 20.0.0

PR #301's QSVCommand.describegpt wrapper unconditionally emitted --json → qsv rejected every real call with Unknown flag: '--json'. The correct flag is --format JSON (qsv 20.0.0 supports Markdown / TSV / JSON / TOON via a single --format flag).

Fix: wrapper now emits --format JSON when json_output=True. Test asserts both presence of --format JSON and absence of --json so this regression can't re-land silently.

Bug 2: _reshape_for_ui assumed wrong output schema

PR #301/#302 assumed qsv emits a flat shape {description, tags, dictionary: [{field, summary}]}. Real qsv 20.0.0 output (validated against LM Studio) is a wrapped envelope with PascalCase top-level keys:

{
  "Dictionary": {
    "response": {"fields": [{"name": "...", "type": "...", "label": "...", "description": "...", ...}], ...},
    "reasoning": "", "token_usage": {...}
  },
  "Description": {
    "response": "<markdown string>",
    "reasoning": "", "token_usage": {...}
  },
  "Tags": {
    "response": {"tags": [...], "attribution": "..."},
    "reasoning": "", "token_usage": {...}, "num_tags": N
  }
}

Even if Bug 1 were fixed, the reshape would always produce an empty dict because none of the assumed top-level keys would match. The UI would never see populated suggestions.

Fix: rewrote _reshape_for_ui to walk the real envelope. Description.response → top-level description; Tags.response.tags → top-level tags (comma-joined); Dictionary.response.fields[i] → per-column entries keyed by name, value = LLM-generated description, label folded into source so the popover surfaces both ("Stock Keeping Unit (SKU)" + long description).

Captured a real qsv envelope as tests/fixtures/qsv_describegpt_sample.json so the happy-path test now exercises actual qsv output, not a synthetic shape. Future qsv schema changes that break the reshape get caught here.

Wrapper: two new optional params for non-localhost LLMs

api_key and base_url. qsv 20.0.0 refuses to start when the resolved base URL doesn't contain localhost AND no API key is set:

Neither QSV_LLM_BASE_URL nor QSV_LLM_APIKEY environment variables are set.
Either set `--base-url` to an address with "localhost" in it, or set `--api-key`.

For DP+ specifically, the worker container reaches LM Studio / Ollama on the host via host.docker.internal — not localhost — so this triggers every time. Documented workaround: set QSV_LLM_APIKEY=NONE in the worker environment or pass api_key="NONE" per call; qsv accepts NONE as the explicit "this is unauthenticated, get on with it" sentinel.

config_declaration.yaml updates

Operators now have a clear path for both:

• Cloud LLM (OpenAI / OpenRouter / etc.): set QSV_LLM_APIKEY in the worker container's environment.
• Local LLM via container-host hostname (Ollama / LM Studio on host.docker.internal): set QSV_LLM_APIKEY=NONE.

Also clarified that format = "JSON" should be set in the prompt-file — qsv emits Markdown by default, which the stage's JSON parse would reject (stage is non-blocking so it skips silently — easy to miss).

Test plan

• 39 tests/test_ai_suggestions.py tests pass (was 36); +3 for new wrapper params (api_key, base_url, omitted-by-default), +2 for partial-envelope and forward-compat reshape, -2 for the synthetic-shape happy-path that was replaced.
• Full unit suite: 166 (was 163). No regressions.
• End-to-end verified: qsvdp 20.0.0 inside dpp-test container → host.docker.internal:1234/v1 → LM Studio (gemma-4-e4b) → real JSON envelope → reshape → on-disk ai_suggestions with description, tags, sku, name, price, quantity_in_stock entries. Each has {value, source} with label folded into source. This is the first time the feature has actually been confirmed working end-to-end against a real LLM.

Lesson

Mocking subprocess.run with a synthetic JSON shape is a false confidence pattern. The 36 passing unit tests on main couldn't catch either bug because nothing in the test loop ever talked to a real qsv binary. The captured fixture committed here is a partial mitigation; a true integration test (real qsv + a stubbed LLM) would catch more. Tracked as a follow-up.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR fixes two real bugs in the AI suggestions feature (introduced in #301/#302) that were uncovered during end-to-end testing against LM Studio. The unit tests previously used a synthetic mocked JSON shape that didn't match qsv 20.0.0's actual CLI surface or output schema, so the bugs slipped through.

Changes:

• Fix QSVCommand.describegpt to emit --format JSON instead of the non-existent --json flag, and add new optional api_key / base_url params (with NONE sentinel support for non-localhost local LLMs like host.docker.internal).
• Rewrite _reshape_for_ui in AISuggestionsStage to walk qsv's actual wrapped PascalCase envelope (Dictionary/Description/Tags with response sub-objects), filter comma-containing tags, and guard against per-column entries colliding with reserved dataset-level/bookkeeping keys.
• Capture a real qsv envelope as a test fixture and expand tests (partial envelope, forward-compat, label-only, comma-tag filtering, reserved-key collisions); update config_declaration.yaml with cloud-vs-local LLM auth guidance and format = "JSON" requirement.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
ckanext/datapusher_plus/qsv_utils.py	Switch --json → --format JSON; add api_key/base_url params with None-omit semantics; expand docstring with the real envelope shape.
ckanext/datapusher_plus/jobs/stages/ai_suggestions.py	Rewrite _reshape_for_ui to consume the wrapped envelope; add reserved-key guard, comma-tag filter, label-only fallback.
ckanext/datapusher_plus/config_declaration.yaml	Document QSV_LLM_APIKEY (cloud) vs QSV_LLM_APIKEY=NONE (local non-localhost) and the required format = "JSON" prompt-file setting.
tests/test_ai_suggestions.py	Replace synthetic happy-path fixture with real captured envelope; add tests for new params, reserved keys, comma tags, label-only, forward-compat.
tests/fixtures/qsv_describegpt_sample.json	Real qsv 20.0.0 describegpt output captured against LM Studio for use as the happy-path fixture.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.