fix(INE-61): bot-review hardening on Task 129 normalization carrier

[e-fu]

Summary

Follow-up to #10 (Task 129) — addresses the 7 actionable findings CodeRabbit + Codex flagged on the original PR before it merged. All hardening, no behavior change on valid input.

• contract_test: guard digest_inventory_findings/3 against non-map digests so Map.keys/1 no longer crashes the run on a scalar
• contract_test: validate _unresolved_reason in stub_value_findings/3 (must be null or the sentinel "not_yet_derived")
• contract_test: drop nil-clauses in digest_findings/2 and stub_record_findings/3 so explicit "key": null produces a finding instead of silently passing
• contract_test: distinguish :missing_input from :invalid_json / malformed-shape in load_parse_methods_inventory/1 — corrupt JSON or wrong top-level shape now raises with regen instructions instead of silently disabling parse_methods_digest_covers_inventory
• normalization: add @spec on six private helpers per the every-function-gets-a-spec mandate
• schema: validate_v4/1 now descends into /normalization so a caller-provided %{} fails preflight
• exchange_v4.json: tighten _unresolved_reason from open-string to enum ["not_yet_derived"] (closed-vocabulary scaffold sentinel)

Test plan

• mix compile --warnings-as-errors — clean
• mix test.json — 173/173 pass on the four touched modules' test files (normalization_test, contract_test_test, schema_test, pipeline_test)
• mix credo --strict --format json — zero new findings on touched files (only pre-existing TODO(Task NN) annotations)
• CI Harness re-run on the merged PR

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Enhanced validation to catch missing or invalid normalization data, now reporting specific errors instead of silently accepting invalid states.
  • Improved error messages when configuration JSON has malformed content or unexpected structure.

• Schema Updates

  • Enforced stricter validation rules for normalization stub records and their configuration values.

[linear-code]

INE-61

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 201b5674-b944-42c1-ab8f-a7b5d5bf8254

📥 Commits

Reviewing files that changed from the base of the PR and between 5ffe33d and 1583597.

📒 Files selected for processing (4)

• lib/ccxt_extract/contract_test.ex
• lib/ccxt_extract/normalization.ex
• lib/ccxt_extract/schema.ex
• priv/schema/exchange_v4.json

---

📝 Walkthrough

Walkthrough

This PR hardens the v4 normalization scaffold by enforcing stricter data shape validation, adding type annotations to digest builder functions, and improving error handling for malformed input files. Schema contracts now require normalization keys and constrain _unresolved_reason to a sentinel value; contract tests reject nil values where maps are required and validate the sentinel with a dedicated helper; parse-methods loading now raises detailed errors instead of silently accepting malformed JSON.

Changes

Normalization Validation Hardening

Layer / File(s)	Summary
Schema Contracts and Required Keys priv/schema/exchange_v4.json, lib/ccxt_extract/schema.ex	_unresolved_reason now restricted to nil or "not_yet_derived" via enum; required normalization keys (parse_methods_digest, field_maps, response_envelopes) enforced in v4 validator.
Private Function Type Specifications lib/ccxt_extract/normalization.ex	Type specs added to digest builder helpers (digest_from_entry, digest_from_methods, digest_record, normalize_params, normalize_param, normalize_statement_count) without changing runtime behavior.
Stricter Contract Test Validation lib/ccxt_extract/contract_test.ex	Normalization digest and stub records no longer accept nil (rejected with "must be a map" finding); _unresolved_reason validated only as nil or "not_yet_derived" via new helper; stub value checks refactored into parser_findings accumulator; digest inventory defensively handles non-map types; load_parse_methods_inventory/1 raises with detailed message for malformed JSON or unexpected top-level shape.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• ZenHive/ccxt_extract#10: Shares overlapping changes to normalization schema, stub validation, and contract test invariants for the same feature.

Poem

🐰 The schemas tighten, boundaries made clear,
No nil shall slip through here—
Types spell what shapes must be,
Validation finds what we decree,
A scaffold stronger than before! ✨

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/task-129-normalization-carrier-293e

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Hardens the v4 “normalization” carrier introduced in Task 129 by tightening schema + preflight validation, adding a dedicated normalization builder, and expanding contract/integration tests to prevent silent drift or crashes on malformed inputs.

Changes:

• Introduces CcxtExtract.Normalization to build a compact parse_methods_digest and scaffold field_maps / response_envelopes, and wires it into the v4 pipeline/schema path.
• Extends v4 preflight validation (Schema.validate_v4/1) and the v4 JSON schema to require and validate the normalization block shape (including _unresolved_reason enum tightening).
• Adds/updates contract, pipeline, and cached integration tests to assert presence, shape, provenance pointers, and digest coverage behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/integration/cached/schema_v4_emit_cached_test.exs	Asserts v4 cached emit includes populated normalization scaffold and sentinel values.
test/ccxt_extract/pipeline_test.exs	Updates v4 expectations around normalization, adds digest projection + provenance assertions, and confirms v3 output omits normalization.
test/ccxt_extract/normalization_test.exs	New unit tests for digest projection, scaffold shape, and defensive behavior.
test/ccxt_extract/contract_test_test.exs	Adds coverage for new normalization invariants and parse_methods digest inventory coverage behavior.
priv/schema/exchange_v4.json	Defines strict Normalization schema and related $defs, including _unresolved_reason enum.
lib/ccxt_extract/schema.ex	Emits normalization in v4 and validates required normalization keys in validate_v4/1.
lib/ccxt_extract/provenance.ex	Marks normalization pointers as derived in default provenance.
lib/ccxt_extract/pipeline.ex	Builds v4 normalization from discovery data and threads it into v4 schema build opts.
lib/ccxt_extract/normalization.ex	New module implementing normalization block derivation and helper APIs for invariants/tests.
lib/ccxt_extract/contract_test.ex	Adds normalization shape invariant, digest coverage invariant, and parse_methods inventory loader.

Comments suppressed due to low confidence (1)

lib/ccxt_extract/normalization.ex:66

• build/2 is implemented defensively (it handles non-map entries via digest_from_entry/1) and your tests call Normalization.build("not a map"), but the spec only allows map() | nil. This mismatch can lead to Dialyzer warnings and misleads callers about accepted input. Consider widening the spec to term() (or adding a guard that enforces map() | nil and updating tests accordingly).

  @spec build(map() | nil, keyword()) :: map()
  def build(parse_methods_entry, _opts \\ []) do
    %{

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

💡 Codex Review

ccxt_extract/lib/ccxt_extract/contract_test.ex

Line 1447 in c9d0ddd

digest = get_in(exchange, ["normalization", "parse_methods_digest"]) || %{}

Guard normalization before descending into it

When a v4 artifact has a malformed non-map normalization value (for example a corrupt file with "normalization": "bad"), run_all/1 now runs check_normalization_shape_valid/2 and then this invariant; the first would report a finding, but this get_in/2 descends through the string and raises instead of returning the contract report. Please fetch/validate the normalization block as a map before reading parse_methods_digest, matching the defensive behavior added for malformed normalization shapes.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".