Skip to content

Allow field deletion on SaaS datasets#7277

Merged
Linker44 merged 8 commits intomainfrom
ENG-2247-allow-field-deletion-on-saas-datasets
Feb 4, 2026
Merged

Allow field deletion on SaaS datasets#7277
Linker44 merged 8 commits intomainfrom
ENG-2247-allow-field-deletion-on-saas-datasets

Conversation

@Linker44
Copy link
Contributor

@Linker44 Linker44 commented Jan 30, 2026

Ticket ENG-2247

Description Of Changes

Previously if a customer deleted a field (that was not added by them) from the dataset our merge logic would revert that change. This was because we didnt want customers breaking their configuration since many dataset fields are referenced in the saas config and are needed for traversal.

On merge logic we now iterate through the whole saas_config and grab any dataset reference we can find. This is later used as a safeguard to prevent deletion of "protected fields". Customers are now able to delete any field as long as its not protected.

Code Changes

  • added logic to preserve fields that are referenced in saas config upon customer deletion of the dataset field

Steps to Confirm

  1. Set up a SaaS integration.
  2. use the endpoint /api/v1/connector-templates/{connector_template_type}/config to see the saas config.
  3. use the endpoint /api/v1/connection/{connection_key}/dataset to edit the dataset, delete a field that is used as dataset reference on the saas config, delete a field that isnt referenced by the saas config.
  4. bump the version of the config in your codebase to trigger an integration update.
  5. verify using the rest api or DBM to verify that the field referenced in the saas config was preserved on the dataset and the other one was deleted.

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • All UX related changes have been reviewed by a designer
    • No UX review needed
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

@Linker44 Linker44 self-assigned this Jan 30, 2026
@vercel
Copy link
Contributor

vercel bot commented Jan 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated (UTC)
fides-plus-nightly Ignored Ignored Preview Feb 3, 2026 5:55pm
fides-privacy-center Ignored Ignored Feb 3, 2026 5:55pm

Request Review

@Linker44 Linker44 requested a review from galvana February 2, 2026 15:35
@Linker44 Linker44 marked this pull request as ready for review February 2, 2026 15:35
@Linker44 Linker44 requested a review from a team as a code owner February 2, 2026 15:35
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 2, 2026

Greptile Overview

Greptile Summary

This PR implements field deletion preservation for SaaS datasets by preventing deletion of fields that are referenced in the SaaS configuration.

Key Changes:

  • Added get_saas_config_referenced_fields() function that recursively traverses SaaS config to find all dataset field references in param_values and postprocessors
  • Modified merge_datasets() and _merge_fields() to accept a protected_fields parameter
  • Fields referenced in SaaS config are now protected from customer deletion to prevent breaking traversal logic
  • Customers can still delete fields that aren't referenced in the SaaS config
  • Integration updates can always delete fields (controlled process)

Testing:

  • Comprehensive parametrized tests cover various SaaS config reference patterns
  • Tests validate protected field behavior across single and multiple collections
  • Existing test expectations updated to reflect new deletion behavior

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The implementation is well-structured with comprehensive test coverage. The recursive traversal approach for finding field references is maintainable and automatically adapts to schema changes. The logic correctly distinguishes between customer deletions (conditionally allowed) and integration deletions (always allowed).
  • No files require special attention

Important Files Changed

Filename Overview
src/fides/service/connection/merge_configs_util.py Adds field reference extraction logic to identify and protect SaaS config-referenced fields from deletion
src/fides/service/connection/connection_service.py Extracts protected fields from SaaS config and passes them to merge_datasets to prevent deletion
tests/service/connection/test_merge_configs_util.py Comprehensive test coverage for field reference extraction from various SaaS config locations

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@@ -0,0 +1,4 @@
type: Changed
description: Customers field deletion are now preserved when merging datasets.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammatical error: "Customers field deletion are" should be "Customer field deletions are"

Suggested change
description: Customers field deletion are now preserved when merging datasets.
description: Customer field deletions are now preserved when merging datasets.

Copy link
Contributor

@galvana galvana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid! Nice tests

@Linker44 Linker44 added this pull request to the merge queue Feb 4, 2026
Merged via the queue into main with commit cd5e0f0 Feb 4, 2026
54 of 55 checks passed
@Linker44 Linker44 deleted the ENG-2247-allow-field-deletion-on-saas-datasets branch February 4, 2026 12:21
@greptile-apps greptile-apps bot mentioned this pull request Feb 6, 2026
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants