Skip to content

fix(describegpt): honor QSV_LLM_BASE_URL env var; repair stale describegpt tests#3889

Merged
jqnatividad merged 1 commit into
masterfrom
describegpt-test-fixes
May 22, 2026
Merged

fix(describegpt): honor QSV_LLM_BASE_URL env var; repair stale describegpt tests#3889
jqnatividad merged 1 commit into
masterfrom
describegpt-test-fixes

Conversation

@jqnatividad
Copy link
Copy Markdown
Collaborator

Fixes 5 describegpt integration tests that fail under QSV_TEST_DESCRIBEGPT=1 against a live local LLM. Surfaced while running the test suite for #3888; unrelated to that PR, so split out here.

Real bug — QSV_LLM_BASE_URL was silently ignored

The DEFAULT_BASE_URL constant (https://api.openai.com/v1) had drifted from the actual --base-url docopt default (http://localhost:1234/v1). Two base-URL precedence checks compare flag_base_url against that sentinel to tell "user explicitly passed --base-url" from "docopt default in effect". Because the sentinel never matched the real default, the comparison was always true — describegpt always took the "explicit flag" branch, used the docopt default, and never consulted QSV_LLM_BASE_URL. The env-var precedence branch was dead code.

Fix: realign DEFAULT_BASE_URL (and the default prompt file's base_url) with the docopt default, so the intended explicit --base-url > QSV_LLM_BASE_URL > default precedence works. Added a comment tying the constant to the docopt default so it can't drift again.

Fixes describegpt_baseurl_precedence_env_over_default.

Test repairs

  • describegpt_stats_options_file_prefix / describegpt_both_file_prefixes — the hand-written stats.csv fixture had an empty nullcount cell in both data rows; parse_stats_csv parses nullcount strictly and errored. Filled in the missing 0s.
  • describegpt_empty_dataset — expected success on a header-only CSV, but describegpt's first step (stats) cannot compile statistics for a zero-row file. Changed it to assert the (correct) error.
  • describegpt_score_high_threshold_triggers_retries — accepted only 2 of the retry-path messages; broadened to all of them, since which message appears depends on the non-deterministic refined query.

Verification

Run against a live local LLM (LM Studio, QSV_TEST_DESCRIBEGPT=1):

  • All 5 previously-failing tests now pass.
  • Full describegpt integration suite: 70 passed, no regressions. (describegpt_jsonschema_dictionary fails intermittently under sustained serial load — pre-existing LLM-output flakiness, passes on re-run.)
  • 81 describegpt unit tests pass.

🤖 Generated with Claude Code

Fixes 5 describegpt integration tests that fail under QSV_TEST_DESCRIBEGPT=1
against a live local LLM.

Real bug - QSV_LLM_BASE_URL was silently ignored:
  The `DEFAULT_BASE_URL` sentinel (`https://api.openai.com/v1`) had drifted
  from the actual `--base-url` docopt default (`http://localhost:1234/v1`).
  The base-URL precedence checks compare `flag_base_url` against that sentinel
  to distinguish "user passed --base-url" from "docopt default in effect", so
  the comparison was always true: describegpt always used the docopt default
  and the QSV_LLM_BASE_URL env-var branch was dead code. Realign the constant
  (and the default prompt file's `base_url`) with the docopt default so the
  intended explicit-flag > env-var > default precedence actually works.
  (fixes describegpt_baseurl_precedence_env_over_default)

Test repairs:
  - The hand-written stats.csv fixture shared by describegpt_stats_options_file_prefix
    and describegpt_both_file_prefixes had an empty `nullcount` cell in both
    rows; `parse_stats_csv` parses nullcount strictly. Filled in the missing 0s.
  - describegpt_empty_dataset expected success on a header-only CSV, but
    describegpt's first step (`stats`) cannot analyze a zero-row file. Changed
    it to assert the (correct) error.
  - describegpt_score_high_threshold_triggers_retries accepted only two of the
    retry-path messages; broadened it to all of them, since which one appears
    depends on the non-deterministic refined query.

Verified against a live local LLM: the 5 tests now pass and the full
describegpt suite is regression-free (the one intermittent failure,
describegpt_jsonschema_dictionary, is pre-existing LLM-output flakiness and
passes on re-run).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread src/cmd/describegpt.rs Dismissed
@codacy-production
Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 7 complexity

Metric Results
Complexity 7

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@jqnatividad jqnatividad merged commit 9562c9e into master May 22, 2026
18 checks passed
@jqnatividad jqnatividad deleted the describegpt-test-fixes branch May 22, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants