Skip to content

Fixes a bug when deploying a cube with partitioned columns due to autoflush#1828

Merged
shangyian merged 2 commits intoDataJunction:mainfrom
shangyian:deployments-bug
Mar 6, 2026
Merged

Fixes a bug when deploying a cube with partitioned columns due to autoflush#1828
shangyian merged 2 commits intoDataJunction:mainfrom
shangyian:deployments-bug

Conversation

@shangyian
Copy link
Copy Markdown
Collaborator

@shangyian shangyian commented Mar 6, 2026

Summary

The bug was in _create_cube_node_revision_from_validation_data in the deployment orchestrator:

  1. When deploying a cube with partitioned columns, the code creates new Column objects and attaches a Partition to each one.
  2. It then explicitly called self.session.add(partition), which seems reasonable, but SQLAlchemy's back-populate relationship meant adding partition to the session also silently added node_column to the session.
  3. The code was inside a no_autoflush block, so this was safe while inside it.
  4. But _create_cube_node_revision_from_validation_data returns node_revision without adding it to the session. The caller appends it to a list of revisions and adds it later via session.add_all(revisions)
  5. Once the no_autoflush block exited, those node_columns were stranded in the session with node_revision_id=null, because node_revision (which would have given them their FK value) hadn't been added to the session yet.
  6. The next db query in the deployment loop triggered an autoflush, which tried to insert those orphaned columns, which lead to the NOT NULL violation on node_revision_id

Instead of session.add(partition), we can just assign node_column.partition = Partition(...) directly. Since Column.partition has cascade="all,delete", SQLAlchemy will automatically save the partition when the column is saved. This way node_column never enters the session prematurely; it only gets added when node_revision is added, at which point SQLAlchemy correctly orders the inserts: NodeRevision first (to get its id), then Column (with node_revision_id set), then Partition (with column_id set).

Test Plan

Verified locally

Deployment Plan

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 6, 2026

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit b1f9618
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/69aa43b4aa6e1800084f6b7c

@shangyian shangyian marked this pull request as ready for review March 6, 2026 03:02
@shangyian shangyian merged commit 0e7f94b into DataJunction:main Mar 6, 2026
17 checks passed
@shangyian shangyian deleted the deployments-bug branch March 6, 2026 03:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant