Skip to content

Gate vast allocation create/activate/deactivate on directory presence - #488

Merged
claire-peters merged 10 commits into
masterfrom
cp_storageapis
Aug 1, 2026
Merged

Gate vast allocation create/activate/deactivate on directory presence#488
claire-peters merged 10 commits into
masterfrom
cp_storageapis

Conversation

@claire-peters

@claire-peters claire-peters commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #487 (merged). userquotas alone isn't authoritative about whether a project's directory actually exists on VAST — it's just quota configuration, and PR #487's identity-model fix still risked creating/activating allocations, or leaving stale ones active, based on quota-list membership alone. This adds folders.stat_path as ground truth, per the real VAST API docs the user found:

  • get_vast_directory_stat(path) wraps client.folders.stat_path.post(path=...). Returns the stat dict when present, None on VAST's 503 "not found" response, and re-raises any other failure — an auth/network error must never be mistaken for "directory doesn't exist," since that risks repeating the same mass-wrongful-deactivation bug PR Add sync_vast_allocations management command #487 fixed.
  • sync_allocation_for_vast_quota now only creates a new allocation or activates a pending request when the project's directory (/{resource_url}/C/{project.title}) is confirmed present; otherwise it's skipped and recorded in a new report['directory_missing'] list.
  • deactivate_allocations_with_missing_directory replaces the userquotas-list-membership check: it stat_path-checks every Active allocation on the resource directly, regardless of whether its project appeared in this run's userquotas at all — catches a project that's vanished from the quota list entirely, not just one whose directory is empty.

Test plan

  • manage.py test coldfront.plugins.vast — 26 tests pass (mocked; no live VAST cluster needed)
  • manage.py check — clean
  • Dry-run sync_vast_allocations against real VAST data in a non-prod environment and confirm report['directory_missing'] and report['deactivated'] match expectations before relying on it in production

claire-peters and others added 10 commits July 17, 2026 12:07
…sence

userquotas alone isn't authoritative about whether a project's directory
actually exists on VAST (it's just quota configuration). Use
folders.stat_path, per real VAST API docs the user found, as ground truth:

- get_vast_directory_stat(path) wraps client.folders.stat_path.post(path=...).
  Returns the stat dict when present, None on VAST's 503 "not found" response,
  and re-raises any other failure - an auth/network error must never be
  mistaken for "directory doesn't exist", since that would risk repeating the
  same category of mass-wrongful-deactivation bug as before.
- sync_allocation_for_vast_quota now only creates a new allocation or
  activates a pending request when the project's directory
  (/{resource_url}/C/{project.title}) is confirmed present; otherwise it's
  skipped and recorded in a new report['directory_missing'] list.
- deactivate_allocations_with_missing_directory replaces the userquotas-list-
  membership check: it stat_path-checks every Active allocation on the
  resource directly, regardless of whether its project appeared in this run's
  userquotas at all - more thorough than list membership, which only caught
  projects with existing quota entries, not ones that vanished from the list
  entirely.
- 6 new tests covering the presence/absence gate on create, activate, and
  deactivate, plus get_vast_directory_stat's three response cases directly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
pull_vast_quotas.py predates the project-based identity fix and the
folders.stat_path directory-presence gating - it still creates allocations
unconditionally and deactivates by userquotas-list membership, the same
category of bug sync_vast_allocations was built to fix. Nothing in this repo
schedules it (not present in add_scheduled_tasks.py), so removing it is safe
here, though an external cron/Django-Q schedule calling it directly would
need to be repointed.

- Delete coldfront/plugins/vast/management/commands/pull_vast_quotas.py
- vast/tasks.py: replace the pull_vast_quotas() wrapper with
  sync_vast_allocations(resource_name=None), matching the wrapper pattern
  isilon/tasks.py already uses for its own new-command task
- Clean up now-stale "matching pull_vast_quotas.py's ..." docstring
  references in vast/utils.py and vast/tests.py

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@claire-peters
claire-peters merged commit 1dd7263 into master Aug 1, 2026
2 checks passed
@claire-peters
claire-peters deleted the cp_storageapis branch August 1, 2026 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant