Skip to content

Implement kbagent catalog generate command #15

@padak

Description

@padak

Summary

Implement automated catalog generation for KBC Explorer. Currently the catalog.json and orchestrations.json data files are produced through a manual 7-step process -- this is not reproducible, not schedulable, and slow.

Goal

A single CLI command that produces a complete, schema-valid catalog snapshot:

kbagent catalog generate --output kbc-explorer/
kbagent catalog generate --tiers tiers.yaml --output kbc-explorer/
kbagent catalog generate --dry-run
kbagent --json catalog generate

Detailed Plan

See docs/PLAN-catalog.md for the full implementation plan, including:

  • Architecture (new commands/catalog.py + services/catalog_service.py)
  • Tier configuration (YAML + convention-based auto-detection)
  • Data collection pipeline (4 phases: parallel fetch, orchestration detail, aggregation, assembly)
  • Job statistics aggregation logic
  • Orchestration assembly logic
  • CLI interface (catalog generate, catalog validate, catalog diff)
  • Schema validation via jsonschema
  • New dependencies: jsonschema, pyyaml

Subcommands

Command Purpose
catalog generate Fetch data from all projects, aggregate, validate, write output files
catalog validate Validate existing catalog.json against schema.json
catalog diff Compare two catalog snapshots and show changes

Acceptance Criteria

  1. kbagent catalog generate produces output matching manually generated catalog
  2. Output passes jsonschema validation against schema.json
  3. Full generation for 27 projects completes in under 60 seconds
  4. --dry-run shows expected API call count without making requests
  5. Tier auto-detection works for ir-l0-*, ir-l1-*, ir-l2-* aliases
  6. Error in one project does not block generation for others

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions