Skip to content

ENG-2594: Add received_at column to currentprivacypreferencev2#7696

Merged
Linker44 merged 5 commits intomainfrom
ENG-2594-add-received-at-to-current-privacy-preference
Mar 18, 2026
Merged

ENG-2594: Add received_at column to currentprivacypreferencev2#7696
Linker44 merged 5 commits intomainfrom
ENG-2594-add-received-at-to-current-privacy-preference

Conversation

@Linker44
Copy link
Contributor

@Linker44 Linker44 commented Mar 18, 2026

Ticket ENG-2594

Description Of Changes

_upsert_preferences_saved() compared incoming received_at against updated_at (server write time) on CurrentPrivacyPreference. For backfill scripts sending historical timestamps via the pre-verified endpoint, updated_at is always "now", so every submission was marked as "truly historical" — only the first processed entry became the current preference regardless of actual chronological order.

The fix (in the companion fidesplus PR) compares received_at vs received_at, falling back to updated_at for legacy records. This PR adds the column to store that value.

Migration notes: The migration adds a nullable column with no default — this is a metadata-only operation in PostgreSQL, instant regardless of row count. Existing records will have NULL received_at and will fall back to updated_at until organically updated. No data backfill needed.

Companion PR: https://github.com/ethyca/fidesplus/pull/3260

Code Changes

  • Add nullable received_at column to CurrentPrivacyPreference model
  • Add alembic migration d98cea042828 to add the column to currentprivacypreferencev2

Steps to Confirm

  1. Run alembic upgrade head and verify the received_at column exists on currentprivacypreferencev2
  2. Run alembic downgrade -1 and verify the column is removed
  3. Confirm existing records have NULL received_at

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • No UX review needed
  • Followup issues:
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
  • Documentation:
    • No documentation updates required

Add a nullable received_at column to CurrentPrivacyPreference so the
companion fidesplus fix can compare incoming received_at against the
stored value instead of updated_at (server write time). This fixes
incorrect current preferences when backfill scripts send historical
timestamps via the pre-verified endpoint.
@vercel
Copy link
Contributor

vercel bot commented Mar 18, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
fides-plus-nightly Ready Ready Preview, Comment Mar 18, 2026 10:53pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
fides-privacy-center Ignored Ignored Mar 18, 2026 10:53pm

Request Review

@Linker44 Linker44 marked this pull request as ready for review March 18, 2026 21:20
@Linker44 Linker44 requested a review from a team as a code owner March 18, 2026 21:20
@Linker44 Linker44 requested review from JadeCara and removed request for a team March 18, 2026 21:20
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 18, 2026

Greptile Summary

This PR adds a nullable received_at column to the CurrentPrivacyPreference model (currentprivacypreferencev2 table) with a corresponding Alembic migration. The change is a prerequisite for a fidesplus companion fix that corrects an upsert comparison bug — previously the code compared an incoming received_at against updated_at (server write time), causing backfill scripts with historical timestamps to always make the first-processed entry the "current" preference.

Key observations:

  • The migration is minimal, correct, and reversible. The down_revision chains properly to baa6792fc3f7.
  • The xx_ filename prefix is consistent with the existing convention in this repo for migrations awaiting sequential numbering.
  • The column is appropriately nullable with no server_default, so existing rows will have NULL and no backfill of this table is needed on the fides side.
  • The companion updated_at column carries index=True; for consistency and query performance on large tables, received_at may benefit from an index as well.

Confidence Score: 5/5

  • This PR is safe to merge — it is a minimal, additive schema change with a correct, reversible migration.
  • The change adds a single nullable column with no risk of breaking existing data or queries. The migration correctly chains to the prior revision, the column definition matches the migration, and the nullable=True approach is correct for a column being added to an existing table. The only minor note is that an index on received_at would better align with updated_at for performance.
  • No files require special attention.

Important Files Changed

Filename Overview
src/fides/api/models/privacy_preference.py Adds a nullable received_at column to CurrentPrivacyPreference; change is minimal, well-placed between property_id and updated_at, and consistent with the existing received_at pattern in ConsentReportingMixinV2.
src/fides/api/alembic/migrations/versions/xx_2026_03_18_1800_d98cea042828_add_received_at_to_.py Correctly adds the received_at column as nullable with timezone support; downgrade removes it cleanly. The down_revision (baa6792fc3f7) chains correctly to the prior migration.

Last reviewed commit: "ENG-2594: Add receiv..."

@Linker44 Linker44 added this pull request to the merge queue Mar 18, 2026
Merged via the queue into main with commit a2a4731 Mar 18, 2026
56 of 57 checks passed
@Linker44 Linker44 deleted the ENG-2594-add-received-at-to-current-privacy-preference branch March 18, 2026 23:34
@Linker44 Linker44 added the db-migration This indicates that a change includes a database migration label Mar 19, 2026
@Linker44 Linker44 self-assigned this Mar 19, 2026
Linker44 added a commit that referenced this pull request Mar 19, 2026
Cherry-picked from PR #7696 (main). Down revision updated to
a1b2c3d4e5f7 (jp-alpha head).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

db-migration This indicates that a change includes a database migration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants