Skip to content

fix(dao): AJDA-2973 accept input manifests with both schema and legacy keys#101

Merged
odinuv merged 1 commit into
mainfrom
odin/AJDA-2973
Jun 30, 2026
Merged

fix(dao): AJDA-2973 accept input manifests with both schema and legacy keys#101
odinuv merged 1 commit into
mainfrom
odin/AJDA-2973

Conversation

@odinuv

@odinuv odinuv commented Jun 30, 2026

Copy link
Copy Markdown
Member

Link to issue

https://linear.app/keboola/issue/AJDA-2973/python-component-scope-manifest-schemalegacy-exclusivity-guard-to

Caused by https://linear.app/keboola/issue/AJDA-2714 — input-mapping now writes the native schema node into input table manifests alongside the legacy keys.

Description

TableMetadata.load_table_metadata_from_manifest rejected any manifest containing both the new schema node and the legacy columns/metadata/column_metadata keys, raising:

UserException: Manifest can't contain new 'schema' and old 'metadata'/'column_metadata'/'columns'

That guard is stage-blind. The library never produces such a manifest itself — write_manifest always reconstructs the manifest from the typed TableDefinition and get_attributes_by_stage emits exactly one format — and there is no raw-write path. So the guard could only ever fire on input manifests, where the platform deliberately writes both formats for backward compatibility. The reader already prefers schema and ignores legacy keys when both are present (is_new_manifestreturn_schema_from_manifest), so the guard is removed.

Behavior after the change: when schema is present it remains the source of truth for columns (legacy column_metadata ignored); table-level legacy metadata continues to be loaded as before.

Justification

Every Python component reading typed input tables started failing once input-mapping (AJDA-2714) began emitting both formats. Observed across kds-team.app-custom-python, keboola.wr-airtable, kds-team.wr-generic, kds-team.app-email-smtp-sender.

Plans for Customer Communication

None.

Impact Analysis

Loosens validation only — no manifest the library writes is affected. Unblocks all Python components consuming dual-format input manifests. Not gated by a feature flag. New regression test covers the dual-format input manifest; full suite (123 tests) and ruff pass.

Deployment Plan

Release a tagged version after merge.

Rollback Plan

Revert this PR.

Post-Release Support Plan

None.

…y keys

The exclusivity guard in TableMetadata.load_table_metadata_from_manifest
rejected any manifest containing both the new `schema` node and the legacy
`columns`/`metadata`/`column_metadata` keys. That guard is stage-blind: the
library never produces such a manifest itself (the writer emits exactly one
format), so it only ever fired on input manifests, where the platform now
writes both for backward compatibility. The reader already prefers `schema`
and ignores legacy keys when both are present, so the guard is removed.
@linear

linear Bot commented Jun 30, 2026

Copy link
Copy Markdown

AJDA-2973

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the manifest-reading logic to accept input table manifests that contain both the new schema node and legacy manifest keys, restoring compatibility with platform-produced dual-format manifests.

Changes:

  • Removed the exclusivity guard in TableMetadata.load_table_metadata_from_manifest that rejected manifests containing both schema and legacy keys.
  • Added a regression test ensuring a dual-format manifest is accepted and that legacy table-level metadata is still loaded while legacy column_metadata is ignored when schema is present.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/keboola/component/dao.py Removes the schema-vs-legacy exclusivity check so dual-format input manifests are accepted.
tests/test_dao.py Adds coverage for a manifest containing both schema and legacy keys, asserting expected precedence/loading behavior.
Comments suppressed due to low confidence (1)

src/keboola/component/dao.py:173

  • The guard for skipping malformed legacy column_metadata entries is incorrect: if not metadata.get("key") and metadata.get("value") will still raise KeyError for items missing key when value is falsy (e.g., None/""). Safely extract key/value via .get() and skip when key is missing or value is None.
            for column, metadata_list in manifest.get("column_metadata", {}).items():
                for metadata in metadata_list:
                    if not metadata.get("key") and metadata.get("value"):
                        continue
                    key = metadata["key"]
                    value = metadata["value"]
                    self.add_column_metadata(column, key, value)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@odinuv odinuv requested a review from a team June 30, 2026 09:32

@soustruh soustruh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ദ്ദി(•ᴗ•)

@odinuv odinuv merged commit 49b4c33 into main Jun 30, 2026
3 checks passed
@odinuv odinuv deleted the odin/AJDA-2973 branch June 30, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants