Refactor dimension link validation into NodeSpecBulkValidator#1951
Merged
shangyian merged 10 commits intoDataJunction:mainfrom Apr 4, 2026
Merged
Refactor dimension link validation into NodeSpecBulkValidator#1951shangyian merged 10 commits intoDataJunction:mainfrom
shangyian merged 10 commits intoDataJunction:mainfrom
Conversation
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Previously, dimension link validation was scattered across the deployment orchestrator as several instance methods (e.g.,
validate_dimension_links). However, these methods were interleaved with deployment execution, making them hard to test in isolation and impossible to call without a running orchestrator.What changes:
DimensionLink.parse_join_sqlandbuild_foreign_key_mappingbecome static methods. They previously required an instantiated DimensionLink ORM object (needing node_revision and dimension relationships loaded). Making them static lets validation call them with just the raw string values from a deployment spec, without touching the DB.NodeSpecBulkValidatorgains_validate_dimension_link_specs, which runs join clause SQL parsing and column existence checks for both join and reference link specs as a post-validation pass. It prefetches dimension node columns in one bulk query (_prefetch_dimension_link_nodes) to avoid per-link DB round trips, and gracefully skips dim-side column checks for dimensions being deployed in the same batch.bulk_validate_node_data(the pre-deployment validation step), not during execution.update_ast_column_typesis promoted from a closure insidevalidate_node_datato a module-level function ininternal/validation.py, soNodeSpecBulkValidatorcan call it after parsing to keep column type inference consistent.column_overridesfield so that validation can substitute a proposed column list for a node (instead of loading from DB). This is a prerequisite for dry-run impact preview in later PRs. This is also threaded thru tobulk_validate_node_dataDJParseExceptioninstead of returning None when a struct field key isn't found, preventing silent type inference failures downstream.Test Plan
make checkpassesmake testshows 100% unit test coverageDeployment Plan