Skip to content

Conversation

@adrianviquez
Copy link
Contributor

@adrianviquez adrianviquez commented Jan 26, 2026

Adding certain fields to be nulled as they create fk dependencies for data not worth exporting. Also adds chunking to prevent transaction incorrectness when exporting

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.


Note

Optimizes and hardens owner data export and archive collection.

  • Adds BATCH_SIZE chunking to all queryset .iterator() calls in archive_collector.py and sql_generator.py to process exports/copies in controlled batches
  • Expands NULLIFIED_FIELDS to avoid FK/sensitive data: adds Owner.bot, OwnerProfile.default_org, Repository.fork/bot, and nullifies author on core.Commit and core.Pull
  • Includes reports.RepositoryFlag in FULL_EXPORT_MODELS for always-on export
  • Minor: wires BATCH_SIZE import where needed and uses it in UPSERT generation

Written by Cursor Bugbot for commit 8a5884c. This will update automatically on new commits. Configure here.

@sentry
Copy link

sentry bot commented Jan 26, 2026

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.54%. Comparing base (a8c620e) to head (8a5884c).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...ared/shared/owner_data_export/archive_collector.py 0.00% 5 Missing ⚠️
...s/shared/shared/owner_data_export/sql_generator.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #676   +/-   ##
=======================================
  Coverage   92.54%   92.54%           
=======================================
  Files        1296     1296           
  Lines       47671    47671           
  Branches     1601     1601           
=======================================
  Hits        44118    44118           
  Misses       3244     3244           
  Partials      309      309           
Flag Coverage Δ
apiunit 96.36% <ø> (ø)
sharedintegration 37.47% <0.00%> (ø)
sharedunit 85.80% <0.00%> (ø)
workerintegration 58.59% <ø> (ø)
workerunit 90.33% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codecov-notifications
Copy link

codecov-notifications bot commented Jan 26, 2026

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...ared/shared/owner_data_export/archive_collector.py 0.00% 5 Missing ⚠️
...s/shared/shared/owner_data_export/sql_generator.py 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codspeed-hq
Copy link

codspeed-hq bot commented Jan 26, 2026

CodSpeed Performance Report

Merging this PR will not alter performance

Comparing adrian/nullify-certain-fields (8a5884c) with main (a8c620e)

Summary

✅ 9 untouched benchmarks

@adrianviquez adrianviquez requested a review from a team January 26, 2026 23:14
@adrianviquez adrianviquez added this pull request to the merge queue Jan 27, 2026
Merged via the queue into main with commit 9a406fd Jan 27, 2026
51 of 53 checks passed
@adrianviquez adrianviquez deleted the adrian/nullify-certain-fields branch January 27, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants