Skip to content

feat: auto-generate Pydantic models from JSON schema & add CI + linting#13

Merged
cka-y merged 8 commits into
mainfrom
feat/match-schema-update
Jun 4, 2026
Merged

feat: auto-generate Pydantic models from JSON schema & add CI + linting#13
cka-y merged 8 commits into
mainfrom
feat/match-schema-update

Conversation

@cka-y
Copy link
Copy Markdown
Contributor

@cka-y cka-y commented Jun 2, 2026

Summary:

Automates diff output model generation and adds CI infrastructure.

  • Add generate_models.sh script to auto-generate Pydantic v2 models from the GTFS Diff JSON Schema using datamodel-code-generator
  • Add schema.conf to pin the schema version (v2-rc1)
  • Add CI workflow with path-filtered jobs: tests (Python 3.10/3.12/3.14), lint check, and models-freshness guard
  • Add Ruff linting & formatting config with lint.sh / lint-fix.sh scripts
  • Update engine logic to align with the v2-rc1 schema
  • Apply lint formatting fixes across cli.py, gtfs_definitions.py, and engine.py

Closes #8, closes #9

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is used to automatically generate the Pydantic models

Comment thread src/gtfs_diff/cli.py
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes are only lint related

Comment thread src/gtfs_diff/engine.py
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes here are lint related + updating some of the logic to match the newly updated v2-rc1 schema. No additional computation logic was added.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint updates exclusively

@cka-y cka-y changed the title feat: automatic pydantic model generation from json schema feat: auto-generate Pydantic models from JSON schema & add CI + linting Jun 2, 2026
@cka-y cka-y marked this pull request as ready for review June 2, 2026 18:55
@cka-y cka-y requested a review from Copilot June 2, 2026 18:56
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modernizes the project’s GTFS Diff v2 output handling by switching to schema-driven (auto-generated) Pydantic v2 models, aligning engine output with the v2-rc1 schema, and adding CI + Ruff-based lint/format tooling to keep models and style consistent.

Changes:

  • Add scripts/generate_models.sh + schema.conf to generate and pin src/gtfs_diff/models.py from the upstream JSON Schema.
  • Update engine + tests to match the updated schema structure (notably moving per-file counts into file_diffs[].stats).
  • Add Ruff configuration/scripts and a path-filtered GitHub Actions workflow (tests, lint, models freshness).

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/gtfs_diff/models.py Replace hand-written models with auto-generated schema-derived Pydantic v2 models.
scripts/generate_models.sh Add a generator script to fetch schema + run datamodel-codegen and post-process the output.
schema.conf Pin the schema version used for generation (v2-rc1).
src/gtfs_diff/engine.py Align output to new schema (introduce FileStats, compute totals from stats, read schema version).
src/gtfs_diff/cli.py Ruff-driven formatting and CLI output formatting tweaks.
src/gtfs_diff/gtfs_definitions.py Formatting-only changes and wrapped docstring.
tests/test_engine.py Update assertions to use file_diffs[].stats and apply formatting.
tests/test_models.py Update/extend model tests for new schema fields (FileStats, files_not_compared_count, etc.).
tests/test_cli.py Formatting updates for fixture zips and imports.
tests/conftest.py Minor whitespace cleanup.
scripts/lint.sh / scripts/lint-fix.sh Add lint/format check and auto-fix scripts via Ruff.
pyproject.toml Add Ruff + datamodel-code-generator (dev deps) and configure Ruff.
.github/workflows/ci.yml Add CI jobs for lint, tests (matrix), and model freshness enforcement.
Comments suppressed due to low confidence (1)

src/gtfs_diff/engine.py:480

  • This comment is now inaccurate: per-file counts are no longer surfaced in summary.files (FileSummary only contains file_name/status). The counts are now in file_diffs[].stats.
    # Determine row-changes output based on cap.
    # cap=0 means "summary counts only" — row_changes is omitted from the output
    # entirely (serialized as null, excluded by exclude_none=True in the CLI).
    # True counts are still computed and surfaced in summary.files.
    base_header_set = set(base_headers)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/gtfs_diff/engine.py
Comment thread scripts/generate_models.sh
Comment thread src/gtfs_diff/cli.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Comment thread src/gtfs_diff/engine.py
Comment thread tests/test_engine.py
Comment thread src/gtfs_diff/engine.py Outdated
Comment thread scripts/generate_models.sh
@cka-y cka-y requested a review from jcpitre June 4, 2026 16:53
Copy link
Copy Markdown
Contributor

@jcpitre jcpitre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@cka-y cka-y merged commit 1ab527f into main Jun 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update schema version to release candidate Support additional properties in diff output models

3 participants