Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change processing of expansions in bulk import scripts to be synchronous #1225

Closed
paynejd opened this issue Mar 1, 2022 · 2 comments
Closed

Change processing of expansions in bulk import scripts to be synchronous #1225

paynejd opened this issue Mar 1, 2022 · 2 comments
Assignees
@snyaggarwal
Labels
api2 OCL API v2 bulk-import This is specific to Bulk Import feature

Comments

@paynejd
Copy link
Member

paynejd commented Mar 1, 2022

Asynchronous processing of expansions can lead to inconsistent results inside a bulk import. Here is an example bulk import script:

  1. Create source
  2. Create concepts
  3. Create collection+auto-expansion with version-less references to those concepts
  4. Update concepts
  5. Create new collection+auto-expansion with version-less references to those concepts
  6. Update concepts
  7. Create new collection+auto-expansion with version-less references to those concepts
  8. Update concepts
  9. Create new collection+auto-expansion with version-less references to those concepts

The result- references will end up pointing to the latest concept version at the time of processing, rather than based on the order that they appear in the bulk import script
@snyaggarwal snyaggarwal added api2 OCL API v2 bulk-import This is specific to Bulk Import feature labels Mar 2, 2022
snyaggarwal added a commit to OpenConceptLab/oclapi2 that referenced this issue Mar 7, 2022
@snyaggarwal
OpenConceptLab/ocl_issues#1225 | Parallel Importer | making sure repo…
e486ad9 
… versions are processed in right order
snyaggarwal added a commit to OpenConceptLab/oclapi2 that referenced this issue Mar 7, 2022
@snyaggarwal
OpenConceptLab/ocl_issues#1225 | Collection Expansion processing to h…
d5d7b35 
…appen in sync in bulk import
snyaggarwal added a commit to OpenConceptLab/oclapi2 that referenced this issue Mar 7, 2022
@snyaggarwal
OpenConceptLab/ocl_issues#1225 | Reference Import | fixing indexing
b689bbe
@snyaggarwal
Copy link
Contributor

@paynejd this is deployed on QA. Can we do a bit of testing on this?

@paynejd
Copy link
Member Author

paynejd commented Mar 8, 2022

Tested on QA - this works perfectly!

@paynejd paynejd closed this as completed Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@snyaggarwal snyaggarwal

Labels
api2 OCL API v2 bulk-import This is specific to Bulk Import feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@paynejd @snyaggarwal