Optimize the MySQL indexes

[leofeyer]

It is apparently recommended to use indexed columns in the ORDER BY clause to speed up sorting.

Edit: The PR now only adds two pid indexes and optimized a tl_search index.

[m-vo]

If this sorting is only for the back end, this might be overkill. Indexes make lookup fast, but they also make updates/deletes/reorganisation slow. Might need some performance testing to be sure... 🙂

[ausi]

It is apparently recommended to use indexed columns in the ORDER BY clause to speed up sorting.

I’m not sure if this is actually an improvement for us as most of our queries do a SELECT * and according to the docs there are many requirements that have to be met in order to use an index for sorting.

I’d only add those indexes for cases where we can measure an actual performance improvement.

[leofeyer]

It definitely makes a huge difference whether or not the ORDER BY column is indexed or not:

Not indexed

Indexed

[leofeyer]

Not indexed

Indexed

[ausi]

I was not able to reproduce these results on my machine (MySQL 8.0.33).

Also, your example queries do not look like real queries that are used by Contao. We never use the tl_search_index.relevance directly for sorting. And we usually have a WHERE clause with pid=?, id=? or something when querying tl_page (except for in backend maybe?).

[leofeyer]

Yes we do:

contao/core-bundle/contao/library/Contao/Search.php

Line 752 in a0cc1ef

$strQuery .= " ORDER BY relevance DESC";

[ausi]

In this case relevance refers to the result of similarity / vectorLength AS relevance where similarity refers to the matching algorithm that uses the tl_search_index.relevance field. The query looks something like this:

SELECT 
	…, 
	similarity / vectorLength AS relevance 
FROM (
	SELECT 
		tl_search_index.pid AS sid,
		(0 + ((1 + LOG(SUM(match0 * tl_search_index.relevance))) * POW(MIN(match0 * matchedTerm.idf), 2) / 1)) / sqrt(0 + POW(MIN(match0 * matchedTerm.idf) / 1, 2)) AS similarity 
	FROM … 
)
ORDER BY relevance DESC

This query cannot make use of an index on tl_search_index.relevance for sorting the result.

[leofeyer]

I have reduced the number of new indexes to 2 in 09721f0. Please review again.