Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure all DoFns are idempotent #1089

Open
bashir2 opened this issue Jun 7, 2024 · 0 comments
Open

Make sure all DoFns are idempotent #1089

bashir2 opened this issue Jun 7, 2024 · 0 comments
Labels
bug Something isn't working P2:should An issue to be addressed in a quarter or so.

Comments

@bashir2
Copy link
Collaborator

bashir2 commented Jun 7, 2024

One of the requirements of a DoFn in the Beam model is idempotence. There are places in the code that we are breaking this rule, for example this TODO which can cause the same FHIR resource being written into Parquet files multiple times.

Another possible example is when we write resources into another FHIR server. We should make sure that updating the same resources multiple times is properly handled.

This tracking issue is to check/fix all potential idempotent issues.

@bashir2 bashir2 added bug Something isn't working P2:should An issue to be addressed in a quarter or so. labels Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P2:should An issue to be addressed in a quarter or so.
Projects
None yet
Development

No branches or pull requests

1 participant