Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch features in background threads #2701

Merged
merged 21 commits into from Jul 24, 2023
Merged

Fetch features in background threads #2701

merged 21 commits into from Jul 24, 2023

Conversation

uclaros
Copy link
Contributor

@uclaros uclaros commented May 29, 2023

This PR fixes #2211
There were two reasons for the slowness:

  1. Searching was done on the main thread, and the search time depends on totalFields x totalFeatures
  2. Numeric fields are checked using the very slow ~ regex operator, which makes 1. even worse

To mitigate this:

  • layer.getFeatures() is ran on a background thread using QtConcurent::run()
  • searching against numeric fields is changed from using the regex operator ~ which is super slow to the ilike operator.

Drawbacks:
#2045 may be easier to hit, as with adequate number of fields + fields names length sqlite hits a parser stack overflow and we get no results at all.
We can address this on a separate PR by requesting all features from the layer and do the matching ourselves.
In that case, the request will only include the relation related expression and not the search filter expression.

app/featuresmodel.cpp Outdated Show resolved Hide resolved
app/featuresmodel.cpp Outdated Show resolved Hide resolved
app/qml/layers/FeaturesListPageV2.qml Outdated Show resolved Hide resolved
app/featuresmodel.h Outdated Show resolved Hide resolved
app/featuresmodel.cpp Show resolved Hide resolved
@PeterPetrik PeterPetrik changed the title Fetch features in bakground threads Fetch features in background threads Jun 13, 2023
@tomasMizera
Copy link
Collaborator

tomasMizera commented Jul 6, 2023

Hi @uclaros, can you please rebase this PR? You will get a new certificate for iOS and its build should work fine then

@tomasMizera tomasMizera merged commit 6db8cc1 into master Jul 24, 2023
8 of 9 checks passed
@tomasMizera tomasMizera deleted the threaded-searching branch July 24, 2023 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Search can get very slow with lots of features
4 participants