MBS-10923: Fix inconsistent ordering in find_by_collection

[mwiencek]

DISTINCT ON (release.id) is used here because the joins create "duplicate" rows for every release. With DISTINCT ON, only the first row from each set is kept, so the ORDER BY in the inner query is important. If $order_by is say date_year, date_month, date_day, name COLLATE musicbrainz, then after sorting by id we also want it to consistently pick the row with the earliest year, then month, etc. (There will be a different row for each release event.)

In the outer query, we also want to make sure that releases that happen to have the same release date and name (or whatever the configured sort order is) are next ordered by id, otherwise the sort is undefined.

One gotcha here is that $order_by references column aliases that can be used in the outer query, but not the inner one. We have to return a separate $inner_order_by with explicit column references where needed. This is icky but I couldn't come up with anything better. One solution might be to replace all the joins with semi-joins, which would also remove the need for DISTINCT ON, but I haven't evaluated the performance of that.

It's not really feasible to write a test for this that fails consistently, since the order is going to depend on the order PG retrieves things, and that's unlikely to change in a very small test DB.

[reosarevok]

Looks generally reasonable I think. Is there any good way to test this at all?

[ijc]

I don't know if it's helpful but
releases.txt.gz is all the releases in the collection which lead to me reporting MBS-10923.

FWIW I've just now reproduced the issue with the curl runes I gave in the report but against the beta server instead (hitting 'https://beta.musicbrainz.org/ws/2/collection/72357d35-55c7-4525-9c7a-7d02e4377f2e/releases?offset=50&order=id', on the assumption that this fix would end up there before hitting production so knowing that the bug shows up there today I can help validate the fix there if it would be helpful.

I tried to setup a local musicbrainz-docker based env with a replica of the real db to try with but failed. As @mwiencek suggests it probably wouldn't be much use anyway since it's likely the issue needs a fully loaded PG db?

[yvanzo]

Tested at https://test.musicbrainz.org.

[mwiencek]

@ijc Thanks, I'd been able to reproduce it with other collections too. We'll update MBS-10923 once the fix is in beta testing, so if you can test it there once that happens that'd be helpful!