Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SQL Import for Soft Delete and History #3530

Merged
merged 13 commits into from
Oct 17, 2023

Conversation

mikaelweave
Copy link
Contributor

@mikaelweave mikaelweave commented Sep 21, 2023

Description

Adds support for importing versioned and deleted resources. This is especially useful for migrating between FHIR instances.

This PR adds the below behavior:

  • Importing with versionId and lastUpdated will result in all versions from an import input file being imported.
    • Previously some versions were skipped (see below).

This is on top of existing behavior which, while not addressed in this PR, is useful for context.

  • Importing without both versionId and lastUpdated means some versions of the resource may not be imported.
    • The code tries to use the latest resource if lastUpdated exists.
  • Importing deleted resources may be skipped (unless versionId and lastUpdated are specified).

I've added E2E tests to ensure resources are properly imported mostly around new behavior but also the existing behavior.

Related issues

AB #105536

Testing

  • New E2E tests which cover various scenarios of deleted/updated resources (new and existing).
  • Manual testing

FHIR Team Checklist

  • Update the title of the PR to be succinct and less than 65 characters
  • Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
  • Tag the PR with the type of update: Bug, Build, Dependencies, Enhancement, New-Feature or Documentation
  • Tag the PR with Open source, Azure API for FHIR (CosmosDB or common code) or Azure Healthcare APIs (SQL or common code) to specify where this change is intended to be released.
  • Tag the PR with Schema Version backward compatible or Schema Version backward incompatible or Schema Version unchanged if this adds or updates Sql script which is/is not backward compatible with the code.
  • CI is green before merge Build Status
  • Review squash-merge requirements

Semver Change (docs)

Patch|Skip|Feature|Breaking (reason)

@mikaelweave mikaelweave added Enhancement Enhancement on existing functionality. Area-SQL Area related to the SQL Server data provider labels Sep 21, 2023
@mikaelweave mikaelweave added this to the S124 milestone Sep 21, 2023
@mikaelweave mikaelweave added Azure Healthcare APIs Label denotes that the issue or PR is relevant to the FHIR service in the Azure Healthcare APIs Area-Operations Area related to operations. labels Sep 21, 2023
@mikaelweave mikaelweave changed the title [DRAFT] - SQL Import with Soft Delete SQL Import with Soft Delete Oct 3, 2023
@mikaelweave mikaelweave marked this pull request as ready for review October 3, 2023 19:33
@mikaelweave mikaelweave requested a review from a team as a code owner October 3, 2023 19:33
@mikaelweave mikaelweave changed the title SQL Import with Soft Delete Fix SQL Import for Soft Delete and History Oct 4, 2023
PTaladay
PTaladay previously approved these changes Oct 13, 2023
@mikaelweave
Copy link
Contributor Author

@SergeyGaluzo - since I had the environment up for export test, I ran $import with this new code one two separate new DBs - one with release branch image and one with the code from this branch. I ran $import using the same sample DB you provided - difference between both runs is about 0.3% (almost nothing). Passed the test - merging this code.

@mikaelweave mikaelweave merged commit 8dfc09f into main Oct 17, 2023
6 checks passed
@mikaelweave mikaelweave deleted the personal/mikaelw/import-soft-delete branch October 17, 2023 03:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Operations Area related to operations. Area-SQL Area related to the SQL Server data provider Azure Healthcare APIs Label denotes that the issue or PR is relevant to the FHIR service in the Azure Healthcare APIs Enhancement Enhancement on existing functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants