test: make column-tag functional tests rerun-safe#1499
Merged
Conversation
pytest --reruns retries a failed test without tearing down the class-scoped project fixture, so mutated state (in-place schema.yml, column tags surviving CREATE OR REPLACE, a running streaming-table query) leaks into the retry and fails it deterministically. Add a RerunSafeMixin that restores the initial model files and drops the relations a test builds before each attempt; the streaming-table case also drops on teardown so its query can't orphan and cascade into co-located tests on the same xdist worker.
sd-db
force-pushed
the
sd-db/fix/streaming-coltags-test-isolation
branch
from
5fee867 to
001c9a4
Compare
Coverage report
This PR does not seem to contain any modification to coverable code. |
jprakash-db
approved these changes
Jun 5, 2026
sd-db
added a commit
that referenced
this pull request
Jun 8, 2026
The constraint tests in test_incremental_constraints.py rewrite schema.yml in place mid-test and build relations, but are not safe under pytest --reruns: a retry reuses the class-scoped project fixture, so the mutated schema.yml and the relations from the failed attempt leak into the retry and fail it deterministically. A single intermittent error then burns all retries (observed on TestIncrementalRemoveForeignKeyConstraint, which failed its initial run plus both reruns in a nightly). Apply the existing RerunSafeMixin (added in #1499) to the schema.yml-mutating classes so each attempt restores the initial model files and drops the relations it builds; FK-holding relations are dropped before their parents. The two DescribeJsonOn subclasses inherit the mixin from their parents. Test-only; no adapter/runtime changes.
sd-db
added a commit
that referenced
this pull request
Jun 8, 2026
## Problem The constraint tests in `tests/functional/adapter/incremental/test_incremental_constraints.py` rewrite `schema.yml` in place mid-test (via `util.write_file`) and build relations, but they aren't safe under `pytest --reruns`. pytest-rerunfailures retries a failed test **without** tearing down the class-scoped `project` fixture, so the mutated `schema.yml` and the relations created by the failed attempt leak into the retry and fail it deterministically — meaning a single intermittent error burns all retries. Observed on `TestIncrementalRemoveForeignKeyConstraint::test_remove_foreign_key_constraint` in a nightly: an intermittent server `INTERNAL_ERROR` failed the initial attempt, and **both** `--reruns 2` retries then failed because they ran against the un-reset, half-built state (only the corrupted FK-parent kept failing; its siblings kept succeeding). ## Change Apply the existing `RerunSafeMixin` (added in #1499) to the `schema.yml`-mutating constraint classes. Before each attempt its autouse fixture restores the initial model files and drops the relations the test builds (named via `relations_to_reset`). FK-holding relations are listed before their parents so the drop order respects the constraint. The two `...DescribeJsonOn` subclasses inherit the mixin from their parents. Test-only; no adapter/runtime changes. ## Verification - Full `test_incremental_constraints.py` on `databricks_uc_cluster`: **15 passed, 0 failed**. - Deterministic rerun-recovery proof (throwaway): a class that mutates `schema.yml` then fails its first attempt **recovers on rerun with the mixin (`RERUN → PASSED`)** and **fails without it (`RERUN → FAILED`)** — the mixin restores `schema.yml` and drops the relations before the retry. Mirrors #1499's forced-rerun repro. - ruff / ruff-format / mypy pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The column-tag functional tests (
test_column_tags.py,test_snapshot_column_tags.py) aren't safe underpytest --reruns. A failed test is retried without tearing down the class-scopedprojectfixture, so mutated state leaks into the retry and fails it deterministically:schema.yml(the retry's firstdbt runuses the updated model),CREATE OR REPLACEon V1 tables,DIFFERENT_DELTA_TABLE_READ_BY_STREAMING_SOURCEinto co-located tests on the same xdist worker.This was behind the intermittent nightly
Integration Tests (Min-Deps)failures.Change
Add
RerunSafeMixin(intests/functional/adapter/fixtures.py): before each attempt it restores the initial model files and drops the relations a test builds (named via arelations_to_resetfixture). The streaming-table test also drops its relation on teardown so the query can't orphan. Test-only; no runtime/adapter changes.Verification
test_column_tags.py(6 classes) +test_snapshot_column_tags.pypass with--reruns 1.