GBIF searching takes too long when country/year is missing

[villanueval]

This is due to the large size of the database (23M records). We need a better way to do the string matching, or do it in the database, when the string does not have a country and/or year to filter the options.

As an alternative, we can do the matching in chunks of data, but it will be slow.

[ajs6f]

What is being used now? grep or similar?

[villanueval]

I wish. It is using approximate string matching using the R package stringdist (e.g. this line). If the string doesn't have the country where the item was collected, it can't select a subset of the whole database, so it gets 23 million rows and then run the approximate match, which takes about 3 minutes, and the user gets bored.

Another option is to run it as a batch and return the results at a later time/date. But that can wait.