Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed pending email query structure to improve speed #5405

Merged
merged 2 commits into from
Dec 21, 2017

Conversation

howlinghuffy
Copy link
Contributor

@howlinghuffy howlinghuffy commented Dec 1, 2017

Q A
Bug fix? Y
New feature? N
Automated tests included? N
Related user documentation PR URL N
Related developer documentation PR URL N
Issues addressed (#s or URLs) #2449 #3826 #3358
BC breaks? N
Deprecations? N

Description:

This fixes a performance bottleneck when loading the Channels -> Emails page with a large number of contacts in Mautic. The old code uses an inefficient NOT EXISTS query, and the new code uses a more efficient NOT IN query.

See an analysis of the different types of queries in MySQL here

To summarise, the query is going from the form:

SELECT  l.*
FROM    t_left l
WHERE   NOT EXISTS
        (
        SELECT  NULL
        FROM    t_right r
        WHERE   r.value = l.value
        )

to

SELECT  l.*
FROM    t_left l
WHERE   l.value NOT IN
        (
        SELECT  value
        FROM    t_right r
        )

It should be noted that the NOT EXISTS and NOT IN queries are slightly different in how they handle NULL values. However, this is irrelevant in our scenario, as the lead_list_leads.lead_id field to which we are comparing the subqueries IDs are set to NOT NULL, so the results will always be equivalent.

Tests with 500,000 contacts show a 200x speed improvement for this query, from 200.5s down to 1.04s.

Steps to reproduce the bug:

  1. Import 500,000+ customer emails
  2. Add at least one email and assign it to a segment with the 500,000+ customer emails
  3. Visit Channels -> Emails
  4. Experience loading times of 10+ minutes

Steps to test this PR:

  1. Import 500,000+ customer emails
  2. Add at least one email and assign it to a segment with the 500,000+ customer emails
  3. Visit Channels -> Emails
  4. Experience loading times of < 10 seconds

@dbhurley dbhurley self-requested a review December 4, 2017 00:09
@escopecz escopecz added ready-to-test PR's that are ready to test user-experience Anything related to related to workflows, feedback, and navigation labels Dec 4, 2017
@alanhartless alanhartless added this to the 2.12.1 milestone Dec 13, 2017
@alanhartless alanhartless added the pending-test-confirmation PR's that require one test before they can be merged label Dec 19, 2017
@javjim javjim self-assigned this Dec 19, 2017
@javjim
Copy link

javjim commented Dec 21, 2017

working properly thanks!!

@javjim javjim removed their assignment Dec 21, 2017
@javjim javjim added ready-to-commit PR's with 2 successful tests, 1 approval, automated tests and docs and is ready to be merged and removed pending-test-confirmation PR's that require one test before they can be merged ready-to-test PR's that are ready to test labels Dec 21, 2017
@escopecz escopecz merged commit 6f05b41 into mautic:staging Dec 21, 2017
@dbhurley dbhurley removed the ready-to-commit PR's with 2 successful tests, 1 approval, automated tests and docs and is ready to be merged label Dec 21, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
user-experience Anything related to related to workflows, feedback, and navigation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants