Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance of loading job details #57

Closed
ghukill opened this issue Dec 1, 2017 · 2 comments
Closed

Performance of loading job details #57

ghukill opened this issue Dec 1, 2017 · 2 comments
Projects
Prototype

Comments

@ghukill
Copy link
Contributor

ghukill commented Dec 1, 2017

Specifically, benchmark and look at the method get_detailed_job_record_count from CombineJob.

Because it counts records and errors from job, each time, it's conceivable that it will prohibitively slow for large jobs.
@ghukill ghukill created this issue from a note in Prototype (ToDo) Dec 1, 2017
@ghukill ghukill moved this from ToDo to In Progress in Prototype Dec 1, 2017
@ghukill
Copy link
Contributor Author

ghukill commented Dec 1, 2017

This is absolutely related to how successful records vs. errors are selected.

Counting records is virtually instant, but counting post-filer like WHERE document != '' is quite slow.

This might be an indicator that separating successful records from errors might be necessary?

@ghukill
Copy link
Contributor Author

ghukill commented Dec 1, 2017

Closing this in favor of more illuminative issue related to docs/errors separation.

@ghukill ghukill closed this as completed Dec 1, 2017
@ghukill ghukill moved this from In Progress to Finis in Prototype Dec 1, 2017
@ghukill ghukill mentioned this issue Dec 1, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
No open projects
Prototype
Finished
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ghukill