Skip to content

roadmap: 0.3.0 command coverage must reach 100% #55

Description

@axisrow

Roadmap: 0.3.0 command coverage must reach 100%

Goal

Version 0.3.0 cannot ship until command/API coverage is 100% across the supported Yandex Direct CLI surface. This issue is the release roadmap and gate. It links live model gaps, command contract, method correctness, write/integration coverage, and the documentation matrix.

Definition of 100% command coverage for 0.3.0

0.3.0 is releasable only when:

  • scripts/build_api_coverage_report.py reports summary.strict_parity_ok == true (✅ already true on main)
  • scripts/build_api_coverage_report.py reports summary.live_model_parity_ok == true (✅ already true on main)
  • model_gaps.live_discovered_missing_services == [] (✅)
  • model_gaps.live_discovered_missing_methods == 0 (✅)
  • A new schema-level gate summary.schema_parity_ok == true (🟡 introduced by fix: WSDL FieldName enums not validated — default fields diverge from API schema #108) — every get-method's default FieldNames is a subset of the corresponding WSDL *FieldEnum, and every SelectionCriteria default satisfies minOccurs=1
  • Every canonical CLI command in the per-service status below shows ✅ on all four checkboxes (or has a documented n/a rationale)
  • Every mutating command has dry-run payload coverage or a documented exclusion
  • Wire method names are validated wherever command aliases or kebab-case map to Yandex camelCase

Current baseline (snapshot 2026-04-25)

  • WSDL services: 29 (declared = live-discovered = 29)
  • WSDL methods: 112 (declared = live-discovered)
  • Non-WSDL services: 1 (reports, JSON API)
  • Total API services covered: 30
  • summary.strict_parity_ok: ✅ true
  • summary.live_model_parity_ok: ✅ true
  • model_gaps.live_discovered_missing_services: []
  • model_gaps.live_discovered_missing_methods: 0
  • Schema-level validation gate (FieldNames + SelectionCriteria): 🟡 partial — see fix: WSDL FieldName enums not validated — default fields diverge from API schema #108 and per-service status below

Per-method status criteria

For every WSDL operation (and every CLI-only guard) the issue tracks four checkboxes:

  1. CLI command exists — registered group/subcommand, mapped to the right WSDL operation.
  2. Default FieldNames validated against WSDL *FieldEnum (only for get-methods). Source of truth: tests/wsdl_cache/<service>.xml. Driver: fix: WSDL FieldName enums not validated — default fields diverge from API schema #108.
  3. SelectionCriteria required params verified (only for methods that take a SelectionCriteria). Source of truth: minOccurs=1 fields in the WSDL request schema.
  4. Integration test passing in sandbox or prod (latest WRITE_SANDBOX live run: Smoke-тесты WRITE_SANDBOX вживую: 20 PASS, 56 ограничений песочницы, 0 неожиданных ошибок #96).

Service-level emoji rollup:

  • ✅ — every method on this service has all 4 checkboxes ticked (or n/a).
  • 🟡 — at least one method has an outstanding checkbox (typically schema or integration).
  • ❌ — at least one WSDL method is missing from the CLI (none today).

smoke annotation refers to the smoke-matrix category from direct_cli/smoke_matrix.py (SAFE, WRITE_SANDBOX, DANGEROUS).


Service-by-service status

1/30. adextensions 🟡

API service: adextensions · CLI group: adextensions · WSDL ops: 3

2/30. adgroups

API service: adgroups · CLI group: adgroups · WSDL ops: 4

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

3/30. adimages

API service: adimages · CLI group: adimages · WSDL ops: 3

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE

4/30. ads

API service: ads · CLI group: ads · WSDL ops: 9

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • archive · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • moderate · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • unarchive · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

5/30. advideos 🟡

API service: advideos · CLI group: advideos · WSDL ops: 2

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • 🟡 get · smoke: SAFE

6/30. agencyclients 🟡

API service: agencyclients · CLI group: agencyclients · WSDL ops: 5 + 1 CLI-only guard

  • add · smoke: DANGEROUS
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • addPassportOrganization (CLI: add-passport-organization) · smoke: DANGEROUS
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • addPassportOrganizationMember (CLI: add-passport-organization-member) · smoke: DANGEROUS
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • 🟡 get · smoke: SAFE
    • CLI command exists (method parity)
    • Default FieldNames validated against WSDL FieldEnum (fix: WSDL FieldName enums not validated — default fields diverge from API schema #108)
    • SelectionCriteria required params verified
    • Integration test passing (sandbox or prod)
    • ⚠️ Integration test (tests/test_integration.py::TestReadOnlyAgencyClients) skips on HTTP 403 / Access denied for non-agency accounts; live coverage therefore depends on an agency token
  • update · smoke: DANGEROUS
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete (CLI-only guard, no WSDL op) · smoke: DANGEROUS
    • CLI command exists
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • [n/a] Integration test
    • Rationale: CLI guard command: the Yandex Direct API does not support deleting agency clients, so the command aborts with an explicit message.

7/30. audiencetargets

API service: audiencetargets · CLI group: audiencetargets · WSDL ops: 6

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • setBids (CLI: set-bids) · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

8/30. bidmodifiers 🟡

API service: bidmodifiers · CLI group: bidmodifiers · WSDL ops: 4

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • 🟡 get · smoke: SAFE
  • set · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

9/30. bids

API service: bids · CLI group: bids · WSDL ops: 3

  • get · smoke: SAFE
  • set · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • setAuto (CLI: set-auto) · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

10/30. businesses 🟡

API service: businesses · CLI group: businesses · WSDL ops: 1

11/30. campaigns

API service: campaigns · CLI group: campaigns · WSDL ops: 8

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • archive · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • unarchive · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

12/30. changes

API service: changes · CLI group: changes · WSDL ops: 3

  • check · smoke: SAFE
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • checkCampaigns (CLI: check-campaigns) · smoke: SAFE
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • checkDictionaries (CLI: check-dictionaries) · smoke: SAFE
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

13/30. clients

API service: clients · CLI group: clients · WSDL ops: 2

  • get · smoke: SAFE
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

14/30. creatives

API service: creatives · CLI group: creatives · WSDL ops: 2

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE

15/30. dictionaries

API service: dictionaries · CLI group: dictionaries · WSDL ops: 2

  • get · CLI helper aliases: list-names · smoke: SAFE
  • getGeoRegions (CLI: get-geo-regions) · smoke: SAFE
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

16/30. dynamicfeedadtargets

API service: dynamicfeedadtargets · CLI group: dynamicfeedadtargets · WSDL ops: 6

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • setBids (CLI: set-bids) · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

17/30. dynamictextadtargets

API service: dynamictextadtargets · CLI group: dynamicads · WSDL ops: 6

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • setBids (CLI: set-bids) · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

18/30. feeds 🟡

API service: feeds · CLI group: feeds · WSDL ops: 4

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • 🟡 get · smoke: SAFE
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

19/30. keywordbids

API service: keywordbids · CLI group: keywordbids · WSDL ops: 3

  • get · smoke: SAFE
  • set · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • setAuto (CLI: set-auto) · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

20/30. keywords

API service: keywords · CLI group: keywords · WSDL ops: 6

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

21/30. keywordsresearch 🟡

API service: keywordsresearch · CLI group: keywordsresearch · WSDL ops: 2

  • deduplicate · smoke: SAFE
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • 🟡 hasSearchVolume (CLI: has-search-volume) · smoke: SAFE
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • SelectionCriteria required params verified
    • Integration test passing (sandbox or prod)
    • ⚠️ WSDL requires both Keywords and RegionIds

22/30. leads 🟡

API service: leads · CLI group: leads · WSDL ops: 1

23/30. negativekeywordsharedsets

API service: negativekeywordsharedsets · CLI group: negativekeywordsharedsets · WSDL ops: 4

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

24/30. retargetinglists

API service: retargetinglists · CLI group: retargeting · WSDL ops: 4

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

25/30. sitelinks

API service: sitelinks · CLI group: sitelinks · WSDL ops: 3

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE

26/30. smartadtargets 🟡

API service: smartadtargets · CLI group: smartadtargets · WSDL ops: 7

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • 🟡 get · smoke: SAFE
  • resume · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • setBids (CLI: set-bids) · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • suspend · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

27/30. strategies

API service: strategies · CLI group: strategies · WSDL ops: 5

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • archive · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE
  • unarchive · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • update · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)

28/30. turbopages

API service: turbopages · CLI group: turbopages · WSDL ops: 1

29/30. vcards

API service: vcards · CLI group: vcards · WSDL ops: 3

  • add · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • delete · smoke: WRITE_SANDBOX
    • CLI command exists (method parity)
    • [n/a] FieldNames validation
    • [n/a] SelectionCriteria validation
    • Integration test passing (sandbox or prod)
  • get · smoke: SAFE

30/30. reports

API service: reports · CLI group: reports · JSON API (no WSDL)

Coverage policy: contract tests + spec snapshot. Source of truth: tests/reports_cache/spec.json, drift script: scripts/check_reports_drift.py.

  • reports.get · smoke: SAFE
    • CLI command exists
    • Field/Filter contract tracked via tests/reports_cache/spec.json
    • Drift detected by scripts/check_reports_drift.py
    • Integration test passing
  • reports.list-types · smoke: SAFE
    • CLI command exists
    • Integration test passing

Schema validation gate (driven by #108)

Before this issue can be closed, #108 must be merged so that the tests/test_api_coverage.py::test_default_fieldnames_match_wsdl_enum test exists and passes for all services. The test parses every *FieldEnum from tests/wsdl_cache/*.xml and asserts that:

  1. Each default FieldNames value (from direct_cli/utils.py:COMMON_FIELDS and from any hard-coded value inside direct_cli/commands/*.py) is a member of the corresponding WSDL enum.
  2. Each get-method that has minOccurs=1 SelectionCriteria fields in WSDL refuses to build a payload without them (no silent empty-SelectionCriteria requests).

Confirmed remaining schema-level work after #107 (driven by #108):

  • smartadtargets — drop Status, ServingStatus from default fields, add State.
  • adextensions — add State to default fields.
  • businesses — drop Type from COMMON_FIELDS (not in WSDL enum).
  • Add the test_default_fieldnames_match_wsdl_enum test in tests/test_api_coverage.py.

Integration-test correctness work

Independent of WSDL parity, several tests/test_integration.py cases pass the wrong CLI flags (root cause of the original audit complaint):

  • leads get — switch test from --campaign-ids to --turbo-page-ids (matches WSDL SelectionCriteria.TurboPageIds minOccurs=1).
  • advideos get — pass --ids (WSDL marks Ids minOccurs=1).
  • agencyclients — treat live error_code=54 ("not an agency account") as expected outcome, not a failure.

These are issues with the tests, not the CLI. They are listed here because they are the visible symptoms users hit when running pytest -m integration -v.


Roadmap (existing phases — kept intact)

Phase 1 — Make coverage gaps machine-visible (#54)

  • Keep api_coverage_report.json as source of truth for the release gate
  • Expose declared vs live-discovered counts
  • Expose missing services/methods under model_gaps
  • Print values in GitHub Actions coverage summary
  • Keep tests/API_COVERAGE.md as the human-readable matrix

Status: ✅ done.

Phase 2 — Close live-discovered service gaps

Originally tracked the dynamicfeedadtargets (×6 methods) and strategies (×5 methods) live gaps.

Status: ✅ done — verified by the per-service status above.

Phase 3 — Fix known command-correctness blockers (#35, #33)

  • Fix keywordsresearch wire-method names (camelCase via METHOD_NAME_OVERRIDES).
  • Add request/dry-run assertions that catch wrong body["method"].
  • Resolve bidmodifiers toggle API ambiguity.

Status: ✅ done — bidmodifiers.toggle deprecated/removed (see comment & tests/API_ISSUE_AUDIT.md).

Phase 4 — Finish canonical command contract & docs (#42, #44)

  • Finalize canonical command naming rules in README.
  • Single-line examples + typed flags only.
  • Docs/tests do not show removed aliases as canonical.
  • Document intentionally unsupported transport gaps.

Status: 🟡 in progress.

Phase 5 — Complete coverage matrix beyond contract parity (#41, #28)

  • Classify every CLI command/group in tests/API_COVERAGE.md.
  • Tag: contract-only / dry-run/schema covered / integration replay covered / sandbox-limited / intentionally unsupported.
  • Extend integration/write confidence without making CI depend on live mutable state.
  • Sandbox limitations explicit and non-blocking only when documented.

Status: 🟡 in progress.


Release gate checklist

  • pytest -q tests/test_api_coverage.py tests/test_reports_drift.py tests/test_dry_run.py tests/test_cli.py tests/test_comprehensive.py passes
  • python3 scripts/build_api_coverage_report.py reports strict_parity_ok: true
  • python3 scripts/build_api_coverage_report.py reports live_model_parity_ok: true
  • model_gaps.live_discovered_missing_services is empty
  • model_gaps.live_discovered_missing_methods is 0
  • NEW: summary.schema_parity_ok == true after fix: WSDL FieldName enums not validated — default fields diverge from API schema #108 merges
  • Every WSDL service block above shows ✅ at the service-level rollup
  • tests/API_COVERAGE.md lists every canonical command with classification
  • README API coverage and CLI contract match the CLI surface
  • All blocker issues closed or explicitly non-blocking with rationale

Exclusion policy (from earlier comment)

The 0.3.0 100% gate counts the supported live API surface only.

  • If an operation is officially deprecated/removed, absent from live WSDL, or contradicted by the official changelog, it must be recorded in tests/API_ISSUE_AUDIT.md before being excluded.
  • Such an operation is not a missing CLI command and must not block 0.3.0.
  • If official docs and live WSDL disagree, classify as docs-drift and resolve before implementing.

Worked example: bidmodifiers.toggle is deprecated since 2025-11-13; live WSDL exposes only bidmodifiers add/delete/get/set; #33 closed as not-planned and excluded from coverage counts.

Related issues

Body regenerated on 2026-04-25 by scripts/build_issue55_body.py. Edit the script, not the body.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementFeature additionneeds:official-verificationNeeds official docs/changelog/WSDL verification before implementationtrackingTracking/umbrella issue

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions