Skip to content

ENG-3572 - Add cloud_infra_group and cloud_infra_group_assignment tables#8189

Merged
vcruces merged 4 commits into
mainfrom
ENG-3572
May 20, 2026
Merged

ENG-3572 - Add cloud_infra_group and cloud_infra_group_assignment tables#8189
vcruces merged 4 commits into
mainfrom
ENG-3572

Conversation

@vcruces
Copy link
Copy Markdown
Contributor

@vcruces vcruces commented May 14, 2026

Ticket ENG-3250

Description Of Changes

Context: The cloud infra monitor discovers AWS resources (S3, RDS, DynamoDB) and stages them as CloudInfraStagedResource records. Unlike IDP monitors where promotion is 1:1 (one app = one System), cloud infra uses a many-to-many model: N resources can belong to M Systems, since a shared resource (e.g. a common RDS instance) may serve multiple business applications. The spike at fidesplus/docs/plans/cloud_infra_resource_grouping_spike.md covers the full design.

This PR adds the ORM models and migration for resource grouping — the data layer that enables users to group cloud infra resources into logical collections that map to Systems. Subsequent PRs will add the service layer, API endpoints, and promotion logic.

Two new tables:

  • cloud_infra_group — a named group targeting a new System (via draft_system_name) or an existing one (via system_id)
  • cloud_infra_group_assignment — join table linking resources to groups (many-to-many), with a promoted flag per assignment tracking whether each resource has been promoted within that specific group

Code Changes

  • New ORM models CloudInfraGroup and CloudInfraGroupAssignment in fides/api/models/detection_discovery/cloud_infra_group.py
  • Alembic migration creating both tables with indexes and FK cascades
  • Partial unique index on (monitor_config_id, system_id) where system_id IS NOT NULL — a System can only have one group per monitor
  • Updated db_dataset.yml with entries for both tables
  • Exported new models in detection_discovery/__init__.py

Steps to Confirm

  1. Run alembic upgrade head — both tables created successfully
  2. Run alembic downgrade -1 — both tables dropped cleanly

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • All UX related changes have been reviewed by a designer
    • No UX review needed
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated (UTC)
fides-plus-nightly Ignored Ignored Preview May 20, 2026 2:56pm
fides-privacy-center Ignored Ignored May 20, 2026 2:56pm

Request Review

@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.12%. Comparing base (86ff003) to head (430e04b).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8189      +/-   ##
==========================================
+ Coverage   85.10%   85.12%   +0.01%     
==========================================
  Files         669      670       +1     
  Lines       43370    43415      +45     
  Branches     5080     5081       +1     
==========================================
+ Hits        36911    36957      +46     
  Misses       5351     5351              
+ Partials     1108     1107       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vcruces vcruces requested a review from adamsachs May 18, 2026 15:11
@vcruces vcruces marked this pull request as ready for review May 18, 2026 15:11
@vcruces vcruces requested a review from a team as a code owner May 18, 2026 15:11
@vcruces vcruces requested a review from a team as a code owner May 18, 2026 18:58
@vcruces vcruces requested review from lucanovera and removed request for a team May 18, 2026 18:58
@vcruces vcruces removed the request for review from lucanovera May 18, 2026 20:23
Base automatically changed from ENG-3250 to main May 18, 2026 20:36
Copy link
Copy Markdown
Contributor

@adamsachs adamsachs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks good to me. only real question is around the SQLAlchemy event hook - i always feel a bit unsure about that approach. but not sure there's anything else right now that makes more sense. i'd maybe just leave a note in the docstring that it could be worth revisiting as we look to productionize this functionality 👍

Comment thread src/fides/api/models/detection_discovery/cloud_infra_group.py
Comment thread src/fides/api/models/detection_discovery/cloud_infra_group.py
@vcruces vcruces added this pull request to the merge queue May 20, 2026
Merged via the queue into main with commit b9acbc8 May 20, 2026
67 of 69 checks passed
@vcruces vcruces deleted the ENG-3572 branch May 20, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants