Optimization of "not translated into" filter

[trang]

The query that is executed when users displayed sentences not translated into XXX contains a several joins.

SELECT DISTINCT s.id FROM sentences s
  JOIN sentences_translations st ON ( s.id = st.sentence_id )
  JOIN sentences t on ( st.translation_id = t.id )
WHERE s.lang = '$source' AND t.lang = '$target'

We could do without these JOINs because we have the required fields already in the sentences_translations table, in the sentence_lang and translation_lang fields.

SELECT sentence_id FROM sentences_translations
WHERE sentence_lang = '$source'
AND translation_lang = '$target'

For a comparison, on my machine, searching for

• English sentences not translated into French: takes 5 minutes... vs 1.49 s.
• Spanish sentences not translated into English: takes 40 seconds vs. 1.13 s.
  https://gist.github.com/trang/928652c1a56fd9b8bdb8

Now the problem when selecting from the sentences_translations table is that we need to make sure the sentence_lang and translation_lang fields are in sync with the language in the sentences table.

When a user changes the language of a sentence, it doesn't update the sentences_translations table accordingly. I didn't take care of this yet.

I have only added a file containing the queries to fix the sentences_translations table but the last queries are really, really slow to execute. I don't know if there's a solution to make them faster, or if it's better to just retrieve the id's to delete and generate a script with a bunch of DELETE FROM sentences_translations WHERE sentence_id = XXX.

So there are at least 2 things to do before this pull request can be merged:

1. Update the sentences_translations when the language of a sentence changes.
2. Find a faster way to sync the languages in sentences_translations with the languages in sentences.

[jiru]

I have only added a file containing the queries to fix the sentences_translations table but the last queries are really, really slow to execute. I don't know if there's a solution to make them faster, or if it's better to just retrieve the id's to delete and generate a script with a bunch of DELETE FROM sentences_translations WHERE sentence_id = XXX.

I don’t think so. I tried to run these two queries on the dev database (and rolled back after). Each removed 2331 rows. The first one (join on st.sentence_id = s.id) took about 25 seconds to execute, but the second one (join on st.translation_id = s.id) took about 2 minutes to complete. It’s still reasonable for a one-time use, but that difference is rather strange. EXPLAIN doesn’t show any difference.