feat(describegpt): deterministic unique_id Content Type for ALL_UNIQUE fields

[jqnatividad]

Summary

• Adds a unique_id token to the Content Type vocabulary used by describegpt --infer-content-type, classifying fields where every row has a distinct non-null value (primary keys, surrogate keys, sequence numbers).
• Detection is deterministic: generate_code_based_dictionary keys off qsv's <ALL_UNIQUE> frequency sentinel (which frequency emits exactly when cardinality == rowcount), so no separate rowcount lookup is needed. combine_dictionary_entries preserves the pre-set value so code-derived facts always win over the LLM's guess.
• Aligns synthesize's faker map: unique_id is added to NON_FAKER_TOKENS so the vocab-coverage invariant passes and synthesize falls back to type+min/max generation (a dedicated per-row-unique generator is left as a follow-up).

Test plan

• cargo test -F all_features --bin qsv — 194 unit tests pass, including 3 new dictionary tests:
  • generate_marks_all_unique_field_as_unique_id
  • generate_skips_unique_id_when_infer_content_type_off
  • combine_preserves_preset_unique_id_over_llm_value
• cargo test -F all_features --test tests test_describegpt — 64 integration tests pass.
• cargo test -F all_features --test tests test_synthesize — 12 integration tests pass.
• cargo clippy --bin qsv -F all_features — clean on touched files.
• qsv --generate-help-md re-run; only docs/help/describegpt.md updated, reflecting the new help text.

🤖 Generated with Claude Code

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 17 complexity

Metric	Results
Complexity	17

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[Copilot]

Pull request overview

Adds deterministic unique_id handling to describegpt --infer-content-type and aligns synthesize’s content-type fallback behavior.

Changes:

• Adds unique_id to the describegpt Content Type vocabulary.
• Pre-sets unique_id for fields with <ALL_UNIQUE> frequency rows and preserves that value over LLM output.
• Updates synthesize non-faker handling and generated help text.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/cmd/describegpt/dictionary.rs	Adds unique_id vocabulary, deterministic dictionary generation, merge behavior, and unit tests.
src/cmd/describegpt.rs	Passes --infer-content-type state into dictionary generation and updates CLI help.
src/cmd/synthesize/faker_map.rs	Treats unique_id as a non-faker token.
resources/describegpt_defaults.toml	Updates the LLM prompt guidance for unique_id.
docs/help/describegpt.md	Updates generated help documentation for the new behavior.

Comments suppressed due to low confidence (1)

src/cmd/describegpt/dictionary.rs:485

• This check treats any frequency row whose value is literally <ALL_UNIQUE> as the sentinel. A real column whose repeated value is <ALL_UNIQUE> would be mislabeled unique_id, while a user-supplied --freq-options --all-unique-text ... sentinel would be missed; include the sentinel semantics (for example rank/percentage plus cardinality == count) instead of only exact text.

        let content_type =
            if infer_content_type && field_frequencies.iter().any(|f| f.value == "<ALL_UNIQUE>") {

[jqnatividad]

Re: Copilot's suppressed (low-confidence) comment on dictionary.rs:485

The comment flagged a real issue worth fixing even though Copilot suppressed it: the original detection — field_frequencies.iter().any(|f| f.value == "<ALL_UNIQUE>") — was text-coupled to frequency's default sentinel and could produce both false positives and false negatives.

Replaced with a structural check in d727221d3 (no literal text matching):

let is_all_unique = stats_record.cardinality > 1
    && stats_record.nullcount == 0
    && field_frequencies.len() == 1
    && field_frequencies[0].count == stats_record.cardinality;

This invariant is exactly what qsv frequency guarantees for ALL_UNIQUE rows (one row, count == row_count == cardinality), so it now:

Scenario	Old check	New check
Genuine ALL_UNIQUE field	✓	✓
HIGH_CARDINALITY (same single-row, 100% shape)	matched (bug)	excluded — cardinality < count
Constant field with value literally <ALL_UNIQUE>	false positive	excluded — cardinality == 1
Mixed field with <ALL_UNIQUE> as one bucket	false positive	excluded — len() > 1
Custom frequency --all-unique-text sentinel	false negative	detected — text-independent
Field with nulls but unique non-nulls	relied on freq to skip	explicitly rejected

4 tests added covering each rejection path plus the custom-sentinel-text acceptance path:

• generate_does_not_mislabel_constant_field_with_all_unique_value
• generate_detects_unique_id_with_custom_all_unique_text
• generate_does_not_misclassify_high_cardinality_as_unique_id
• generate_does_not_mislabel_unique_id_when_nulls_present