Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Processing's batch process performs badly when adding a large number of files #38987

Closed
nirvn opened this issue Sep 24, 2020 · 3 comments 路 Fixed by #38989
Closed

Processing's batch process performs badly when adding a large number of files #38987

nirvn opened this issue Sep 24, 2020 · 3 comments 路 Fixed by #38989
Labels
Bug Either a bug report, or a bug fix. Let's hope for the latter! Processing Relating to QGIS Processing framework or individual Processing algorithms

Comments

@nirvn
Copy link
Contributor

nirvn commented Sep 24, 2020

When auto-filling the batch table with a large number of files, QGIS will freeze for a long time (e.g. on one machine, 3000 files = +/-20 minutes 馃槺 ).

Here's a hotspot graph output that indicates a couple of bottlenecks:
image
@nirvn nirvn added Bug Either a bug report, or a bug fix. Let's hope for the latter! Processing Relating to QGIS Processing framework or individual Processing algorithms labels Sep 24, 2020
@nyalldawson nyalldawson mentioned this issue Sep 24, 2020
Merged
@nirvn nirvn mentioned this issue Sep 24, 2020
Merged
@nirvn
Copy link
Contributor Author

nirvn commented Sep 24, 2020

Slow down github, we still need @nyalldawson 's magic to consider it fixed.

@nirvn nirvn reopened this Sep 24, 2020
nyalldawson added a commit to nyalldawson/QGIS that referenced this issue Sep 24, 2020
@nyalldawson
Only build list of supported raster/vector filters and formats once
3f4c0c8 
It's expensive to calculate, so only ever do it once...

Refs qgis#38987
nyalldawson added a commit that referenced this issue Sep 25, 2020
@nyalldawson
Only build list of supported raster/vector filters and formats once
776768e 
It's expensive to calculate, so only ever do it once...

Refs #38987
@nyalldawson
Copy link
Collaborator

@nirvn can we close?

@nirvn
Copy link
Contributor Author

nirvn commented Oct 7, 2020

Yep

@nirvn nirvn closed this as completed Oct 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Either a bug report, or a bug fix. Let's hope for the latter! Processing Relating to QGIS Processing framework or individual Processing algorithms
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make processing's batch process faster nirvn/QGIS
2 participants
@nirvn @nyalldawson