Fix migration 0156 CheckViolation by normalizing invalid base_path values - #8068
Open
dkliban wants to merge 6 commits into
Open
Fix migration 0156 CheckViolation by normalizing invalid base_path values#8068dkliban wants to merge 6 commits into
dkliban wants to merge 6 commits into
Conversation
…e cast (issue pulp#8067) Migration 0156 fails with a CheckViolation when any existing row in core_distribution has a base_path value that doesn't satisfy the relative_path domain constraint introduced in 0155. Add a RunPython step before the AlterField operation that: - Finds all rows whose base_path values would fail the constraint - Normalizes them (strips whitespace, leading/trailing slashes, double slashes, and bare dot/dotdot components) - Raises a RuntimeError with the offending rows listed if any path cannot be automatically fixed, allowing administrators to correct the data manually before retrying the migration Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…6 fix
- Remove unused 'import django.db.models.expressions' (ruff I001)
- Add unit tests for _normalize_path and fix_base_path_violations:
* Parametrized tests covering trailing/leading slashes, whitespace,
query strings, fragments, dot/dotdot components, and clean paths
* Mock-based tests for the database update path and unfixable path
error handling
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…isort config ruff isort treats pulpcore as 'second-party' with its own import group after third-party packages (django.*). Separate it from the Django imports with a blank line to satisfy the I001 rule. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Remove unused 'call' import (ruff F401) - Remove unused 'pk'/'clean_path' variables from skips-clean-rows test (ruff F841) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Join SQL string onto single line (ruff format prefers no implicit concatenation) - Remove blank line after class declaration (ruff format) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
ruff format wraps lines >88 chars; the patch() calls in tests had very long module paths that needed to be reformatted. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migration
core.0156_alter_contentartifact_relative_path_and_morefails with aCheckViolationwhen any existingcore_distribution.base_pathvalue does not satisfy therelative_pathdomain check constraint introduced in migration 0155.Reproduces as reported in #8067: after a failed upgrade to 3.117.0 (deadlock during migration), the retry with 3.117.1 fails with:
Root cause:
AlterFieldin migration 0156 changescore_distribution.base_pathfromtextto therelative_pathdomain. PostgreSQL validates all existing rows against the domain's check constraint during this operation. Any path value containing whitespace,?,#, double-slashes, leading/trailing slashes, or dot/dotdot components violates the constraint and causes the migration to abort.Fix: Add a
RunPythonstep before theAlterFieldoperation that:base_pathviolates the constraint (using the same regex as the domain)RuntimeErrorlisting any paths that cannot be automatically fixed, so administrators can correct them before retryingTests: Unit tests for
_normalize_path(parametrized, no DB required) and forfix_base_path_violations(mock-based, tests the normalize, skip-clean, and unfixable-path code paths).Closes #8067
Test plan
pytest pulpcore/tests/unit/models/test_0156_migration.pypassesbase_pathvaluesbase_pathvalues (trailing slash, whitespace, query string)base_pathvalues (e.g. bare.)🤖 Generated with Claude Code