Skip to content

fix(versioning): SAVEPOINT-isolate baseline capture so a failure can't poison the save#41910

Closed
mikebridge wants to merge 2 commits into
apache:masterfrom
mikebridge:fix-versioning-baseline-savepoint
Closed

fix(versioning): SAVEPOINT-isolate baseline capture so a failure can't poison the save#41910
mikebridge wants to merge 2 commits into
apache:masterfrom
mikebridge:fix-versioning-baseline-savepoint

Conversation

@mikebridge

@mikebridge mikebridge commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Depends on #41176 (versioning base infrastructure) — do NOT merge until that lands. This fix touches superset/versioning/baseline/insertion.py, which is introduced by #41176, so this branch is stacked on it; the diff below includes #41176's changes until it merges, at which point this rebases to a one-line diff. Kept as a draft until then.

SUMMARY

Parent baseline capture in superset/versioning/baseline/insertion.py runs its direct-SQL inserts under try/except + no_autoflush but not a SAVEPOINT. On PostgreSQL a failed statement aborts the entire transaction even when the exception is caught, so a baseline-insert failure would poison the user's save — contradicting the "versioning must never break a save" guarantee this subsystem is built around.

This wraps the inserts in session.connection().begin_nested(), so a failure rolls back to the savepoint and the outer transaction stays healthy — degrading to "no baseline row" instead of a broken save. It mirrors the guard the change-record persist path (changes/listener.py) and the child-baseline path (baseline/collection.py:133) already use; the parent-baseline path was the one spot missing it.

The affected path only runs with ENABLE_VERSIONING_CAPTURE on (the base infra ships dark/off), so this is a pre-flag-flip correctness fix, not a deploy-time regression — hence a follow-up rather than a change to #41176 at its merge gate.

Surfaced during a multi-model review pass on #41176.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A — backend transaction-safety fix.

TESTING INSTRUCTIONS

Mirrors an existing, tested pattern (baseline/collection.py). Includes a dedicated fault-injection regression test — test_baseline_failure_does_not_break_the_save in tests/integration_tests/versioning/versions_api_tests.py — which injects a real failing SQL statement into the parent-baseline path during a capture-on chart save and asserts the edit still commits. Without the SAVEPOINT this fails on the PostgreSQL CI job (the aborted transaction poisons the outer save); with it, the baseline rolls back and the save succeeds. It uses real SQL rather than a Python raise because only a failed DB statement poisons a Postgres transaction — so it passes trivially on SQLite, and the Postgres job is the real guard. The existing versioning capture tests cover the happy path.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags: ENABLE_VERSIONING_CAPTURE (the affected code path only runs with capture on)
  • Changes UI
  • Includes DB Migration
  • Introduces new feature or API
  • Removes existing feature or API

🤖 Generated with Claude Code

@github-actions github-actions Bot added risk:db-migration PRs that require a DB migration api Related to the REST API labels Jul 9, 2026
@mikebridge

Copy link
Copy Markdown
Contributor Author

⛔ Blocked on #41176 — this is stacked on the versioning base-infrastructure PR and can't merge until that lands. Keeping it a draft until then; the diff will shrink to the one-line fix once #41176 merges and this rebases onto master.

(Posted via Claude on my behalf.)

@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.78%. Comparing base (2112fbd) to head (44cd2a4).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #41910      +/-   ##
==========================================
- Coverage   64.78%   64.78%   -0.01%     
==========================================
  Files        2739     2739              
  Lines      152981   152981              
  Branches    35058    35058              
==========================================
- Hits        99114    99107       -7     
- Misses      51970    51974       +4     
- Partials     1897     1900       +3     
Flag Coverage Δ
hive 39.08% <0.00%> (ø)
mysql 58.00% <100.00%> (ø)
postgres 58.06% <100.00%> (-0.01%) ⬇️
presto 41.04% <100.00%> (ø)
python 59.45% <100.00%> (-0.01%) ⬇️
sqlite 57.67% <100.00%> (ø)
unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Mike Bridge and others added 2 commits July 10, 2026 09:39
The parent-baseline direct-SQL inserts in baseline/insertion.py ran under
``try/except`` + ``no_autoflush`` but not a SAVEPOINT. On PostgreSQL a
failed statement aborts the whole transaction even when the exception is
caught, so a baseline-insert failure would poison the user's save —
contradicting the "versioning must never break a save" guarantee.

Wrap the inserts in ``session.connection().begin_nested()`` so a failure
rolls back to the savepoint and the outer transaction stays healthy,
degrading to "no baseline row" instead of a broken save. This mirrors the
guard the change-record persist path (changes/listener.py) and the
child-baseline path (baseline/collection.py) already use; the
parent-baseline path was the one spot missing it.

Only reachable with ENABLE_VERSIONING_CAPTURE on (the base infra ships
dark/off), so this is a pre-flag-flip correctness fix, not a deploy-time
regression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inject a real failing SQL statement into the parent-baseline path
(insert_baseline_shadow_row) during a capture-on chart save, and assert the
user's edit still commits. Without the begin_nested() SAVEPOINT this fails on
PostgreSQL — the aborted transaction poisons the outer save; with it, the
baseline attempt rolls back to the savepoint and the save succeeds. Uses a
real SQL error rather than a Python raise because only a failed DB statement
poisons a PostgreSQL transaction (so the test is a genuine guard on the
Postgres CI job; it passes trivially on SQLite, which doesn't poison).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mikebridge mikebridge force-pushed the fix-versioning-baseline-savepoint branch from 26f82a3 to 44cd2a4 Compare July 10, 2026 15:39
@mikebridge mikebridge marked this pull request as ready for review July 10, 2026 15:39
@github-actions github-actions Bot removed risk:db-migration PRs that require a DB migration api Related to the REST API labels Jul 10, 2026
@bito-code-review

bito-code-review Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #a82787

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: a5d94e4..44cd2a4
    • superset/versioning/baseline/insertion.py
    • tests/integration_tests/versioning/versions_api_tests.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@mikebridge

Copy link
Copy Markdown
Contributor Author

Closing in favor of the broader transaction-correctness follow-up tracked in sc-112938.

This PR proves the PostgreSQL transaction-poisoning problem for a failed parent baseline write, but its begin_nested() boundary runs inside the prepended before_flush listener and covers only the parent failure case. Follow-up investigation found that the complete fix also needs to prove parent/child atomic rollback, absence of partial baseline history, continued session usability, and compatibility with SQLAlchemy-Continuum canonical shadow generation.

The replacement work is being developed separately from current master on sc-112938-baseline-fail-open, with PostgreSQL fault-injection tests driving the lifecycle design before production changes. Thanks to the test and analysis here—they remain useful evidence for the replacement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant