Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add queuing feature to all bulk import methods #704

Closed
paynejd opened this issue Apr 9, 2021 · 5 comments
Closed

Add queuing feature to all bulk import methods #704

paynejd opened this issue Apr 9, 2021 · 5 comments
Assignees
Labels
api2 OCL API v2 enhancement New feature or request
Milestone

Comments

@paynejd
Copy link
Member

paynejd commented Apr 9, 2021

The bulk import queue feature was added to the original /manage/bulkimport endpoint, but it is not yet available in the inline/parallel versions. We want this feature to be available in all of the bulk import endpoints.

After this is completed, we can continue work on #613

@snyaggarwal
Copy link
Contributor

@paynejd This should be available now

@snyaggarwal snyaggarwal added api2 OCL API v2 enhancement New feature or request labels May 6, 2021
@paynejd paynejd assigned paynejd and unassigned snyaggarwal May 7, 2021
@paynejd paynejd modified the milestones: Sprint 9, Sprint 10 May 7, 2021
@paynejd
Copy link
Member Author

paynejd commented May 13, 2021

My baseline bulk import for PEPFAR country mappings took ~185sec with the old import method. The bulk import method took just 75seconds!

I did not see queue key anywhere on the bulk import UI, but it used to be there -- was that field hidden? That would be useful to control the queue key still so that we could manage the order in which bulk import files are processed. For example, I may want to submit 10 bulk imports to the same queue key - this would force the files to be processed in the order in which they were submitted. The queue key would be a useful field to display in the list of existing imports as well.

Next step for testing is to make sure that the scripts can get all the info that they need about queued imports, as this is how they determine if an import is in process.

@snyaggarwal
Copy link
Contributor

@paynejd Enabled custom-queue field for every user.
Note: Right now ocladmin and root user's imports will always go to "priority" queue (despite of the fact a custom-queue is defined or not)

snyaggarwal added a commit to OpenConceptLab/oclweb2 that referenced this issue May 14, 2021
@snyaggarwal
Copy link
Contributor

@paynejd The queue key is displayed in existing imports as well.

@paynejd
Copy link
Member Author

paynejd commented May 21, 2021

Initial testing went great - closing this out! Additional testing will take place as part of #613

@paynejd paynejd closed this as completed May 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api2 OCL API v2 enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants