Skip to content

Fix deployment idempotency for dimension links and downstream validation#2032

Merged
shangyian merged 1 commit intoDataJunction:mainfrom
shangyian:deploy-links
Apr 20, 2026
Merged

Fix deployment idempotency for dimension links and downstream validation#2032
shangyian merged 1 commit intoDataJunction:mainfrom
shangyian:deploy-links

Conversation

@shangyian
Copy link
Copy Markdown
Collaborator

@shangyian shangyian commented Apr 20, 2026

Summary

Dimension Link Reshuffling

When a node has multiple links to the same dimension with no role (e.g., two date links with different join_on), the matching logic used (dimension name, role) as the identity key. This caused it to treat distinct links as duplicates (deleting one and updating the other) on every push, which triggered downstream revalidation.

After session.flush() in bulk_deploy_nodes_in_level, SQLAlchemy expired the dimension links collection on new revisions. Later access in _deploy_links triggered a lazy load that silently failed in async context, causing the match to find 0 existing links and create duplicates.

Impact Propagation

Freshly deployed nodes had raw string column types (e.g., 'bigint') that hadn't round-tripped through ColumnTypeDecorator. When impact propagation passed these to validate_node_query, it would fail because it expected parsed ColumnType objects.

Test Plan

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 20, 2026

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit 422fff1
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/69e616293dff7900078bc979

@shangyian shangyian marked this pull request as ready for review April 20, 2026 10:46
@shangyian shangyian changed the title Fix deployment idempotency issues Fix deployment idempotency for dimension links and downstream validation Apr 20, 2026
@shangyian shangyian added the bug Something isn't working label Apr 20, 2026
@shangyian shangyian merged commit 20a6112 into DataJunction:main Apr 20, 2026
25 checks passed
@shangyian shangyian deleted the deploy-links branch April 20, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant