Remove some of the unnecessary distinct

[jonmol]

What does it do?

Instead of always adding distinct if there's a sort or a where clause it's checking if there's a join and not a limit 1 (since then it doesn't matter if they are distinct)

Why is it needed?

It's a temporary workaround for #9531. I suspect fixing that one properly would take some time and this will cover a lot of the cases in the mean time. It doesn't change the behaviour of anything, just making things a bit more efficient.

How to test it?

Run with debug true for the database and use an SQL database

Related issue(s)/PR(s)

#9531

[codecov]

Codecov Report

Merging #9538 (dc2607a) into master (3065d72) will not change coverage.
The diff coverage is n/a.

❗ Current head dc2607a differs from pull request most recent head 1febe2c. Consider uploading reports for the commit 1febe2c to get more accurate results

@@           Coverage Diff           @@
##           master    #9538   +/-   ##
=======================================
  Coverage   35.90%   35.90%           
=======================================
  Files        1342     1342           
  Lines       14812    14812           
  Branches     1477     1477           
=======================================
  Hits         5318     5318           
  Misses       8575     8575           
  Partials      919      919           

Flag	Coverage Δ
front	27.25% <ø> (ø)
unit	55.58% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3065d72...1febe2c. Read the comment docs.

[jonmol]

While this PR isn't solving all the problems, it should speed up these two a lot:

And at least in our case, they stand for very close to all the stress that's being put on the database server.

[jonmol]

Any chance this one could be merged? An example of what I'm seeing with the current distinct:

"Limit  (cost=298095.66..298096.16 rows=10 width=1795) (actual time=276952.664..276952.695 rows=10 loops=1)"
"  ->  Unique  (cost=298095.66..306238.11 rows=162849 width=1795) (actual time=276952.651..276952.679 rows=10 loops=1)"
"        ->  Sort  (cost=298095.66..298502.78 rows=162849 width=1795) (actual time=276952.646..276952.653 rows=10 loops=1)"
"              Sort Key: updated_at DESC, id, name, ""alternativeText"", caption, width, height, formats, hash, ext, mime, size, url, ""previewUrl"", provider, provider_metadata, created_by, updated_by, created_at"
"              Sort Method: external merge  Disk: 197480kB"
"              ->  Seq Scan on upload_file  (cost=0.00..30178.49 rows=162849 width=1795) (actual time=0.024..140.815 rows=162146 loops=1)"
"Planning Time: 0.192 ms"
"Execution Time: 276983.192 ms"

And with this patch it changes to:

"Limit  (cost=0.42..3.65 rows=10 width=1795) (actual time=0.066..0.090 rows=10 loops=1)"
"  ->  Index Scan using idx_upload_updated_at on upload_file  (cost=0.42..52711.15 rows=163192 width=1795) (actual time=0.064..0.087 rows=10 loops=1)"
"Planning Time: 0.257 ms"
"Execution Time: 0.140 ms"

This is with about 165k rows in the upload_file table. With 20-30 people working at the same time it brings our database to its knees.

[gu-stav]

@jonmol I'd also really like to have this merged. Could you work on the two open questions? Otherwise I'm happy to take over, since it's a small change.

[jonmol]

@gustavpursche sorry for being a bit slow, we made a fork with the change so it was fixed locally and fell off the radar a bit.

Changed the if-statements @alexandrebodin

[gu-stav]

@jonmol Sorry, I didn't want to be pushy :) Thanks for the fix!

[jonmol]

No worries @gustavpursche, I needed a nudge.

If you're, like us, having performance problems with the database and Strapi did this and #9316 lower our database load a lot, and our internal staff using Strapi is a lot happier. 9316 didn't have quite as much impact as this one, but taking away search filters on a lot of things gives a fair performance improvement as well. Especially for the upload_data table if you have plenty in it.

[alexandrebodin]

Just some renaming but the logic is good 👍

[alexandrebodin]

Thank you for this improvement

[derrickmehaffy]

This pull request has been mentioned on Strapi Community Forum. There might be relevant details there:

https://forum.strapi.io/t/slow-data-fetch-in-admin-ui-caused-by-select-distinct-in-postgresql/3316/2