Gate vast allocation create/activate/deactivate on directory presence - #488
Merged
Conversation
…sence
userquotas alone isn't authoritative about whether a project's directory
actually exists on VAST (it's just quota configuration). Use
folders.stat_path, per real VAST API docs the user found, as ground truth:
- get_vast_directory_stat(path) wraps client.folders.stat_path.post(path=...).
Returns the stat dict when present, None on VAST's 503 "not found" response,
and re-raises any other failure - an auth/network error must never be
mistaken for "directory doesn't exist", since that would risk repeating the
same category of mass-wrongful-deactivation bug as before.
- sync_allocation_for_vast_quota now only creates a new allocation or
activates a pending request when the project's directory
(/{resource_url}/C/{project.title}) is confirmed present; otherwise it's
skipped and recorded in a new report['directory_missing'] list.
- deactivate_allocations_with_missing_directory replaces the userquotas-list-
membership check: it stat_path-checks every Active allocation on the
resource directly, regardless of whether its project appeared in this run's
userquotas at all - more thorough than list membership, which only caught
projects with existing quota entries, not ones that vanished from the list
entirely.
- 6 new tests covering the presence/absence gate on create, activate, and
deactivate, plus get_vast_directory_stat's three response cases directly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pull_vast_quotas.py predates the project-based identity fix and the folders.stat_path directory-presence gating - it still creates allocations unconditionally and deactivates by userquotas-list membership, the same category of bug sync_vast_allocations was built to fix. Nothing in this repo schedules it (not present in add_scheduled_tasks.py), so removing it is safe here, though an external cron/Django-Q schedule calling it directly would need to be repointed. - Delete coldfront/plugins/vast/management/commands/pull_vast_quotas.py - vast/tasks.py: replace the pull_vast_quotas() wrapper with sync_vast_allocations(resource_name=None), matching the wrapper pattern isilon/tasks.py already uses for its own new-command task - Clean up now-stale "matching pull_vast_quotas.py's ..." docstring references in vast/utils.py and vast/tests.py Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #487 (merged).
userquotasalone isn't authoritative about whether a project's directory actually exists on VAST — it's just quota configuration, and PR #487's identity-model fix still risked creating/activating allocations, or leaving stale ones active, based on quota-list membership alone. This addsfolders.stat_pathas ground truth, per the real VAST API docs the user found:get_vast_directory_stat(path)wrapsclient.folders.stat_path.post(path=...). Returns the stat dict when present,Noneon VAST's 503 "not found" response, and re-raises any other failure — an auth/network error must never be mistaken for "directory doesn't exist," since that risks repeating the same mass-wrongful-deactivation bug PR Add sync_vast_allocations management command #487 fixed.sync_allocation_for_vast_quotanow only creates a new allocation or activates a pending request when the project's directory (/{resource_url}/C/{project.title}) is confirmed present; otherwise it's skipped and recorded in a newreport['directory_missing']list.deactivate_allocations_with_missing_directoryreplaces the userquotas-list-membership check: it stat_path-checks every Active allocation on the resource directly, regardless of whether its project appeared in this run's userquotas at all — catches a project that's vanished from the quota list entirely, not just one whose directory is empty.Test plan
manage.py test coldfront.plugins.vast— 26 tests pass (mocked; no live VAST cluster needed)manage.py check— cleansync_vast_allocationsagainst real VAST data in a non-prod environment and confirmreport['directory_missing']andreport['deactivated']match expectations before relying on it in production