New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index User lookup by key to address DB connection pool exhaustion #3584

Merged
merged 1 commit into from Jul 8, 2017

Conversation

Projects
None yet
2 participants
@nilbus
Member

nilbus commented Jul 8, 2017

I noticed in BugSnag that we were still seeing this error:

ActiveRecord::ConnectionTimeoutErrorapi/v1/routes/core.rb:46
could not obtain a connection from the pool within 5.000 seconds (waited 5.567 seconds); all pooled connections were in use

These tracebacks were not originating at many various database calls like I would expect, but rather all at the same one, which indicates to me that this one call is responsible for a large portion of our database time.

        def find_user
          User.where(key: params[:key]).first if params[:key]
        end

Sure enough, there is no index for this common user lookup. Without it, the DB must scan all 150k users for each user lookup by key.

I think this should help with the ongoing connection pool issue! 🏇

@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Jul 8, 2017

Member

I would have made this a unique index, except that we actually have duplicate users in our production database. One example:

# select id, username, key, updated_at from users where key='[redacted]';
  id   |    key     |         updated_at
-------+------------+----------------------------
 14804 | [redacted] | 2014-06-05 11:59:17.267012
 14805 | [redacted] | 2014-06-05 11:59:13.151091

These could be due to a race condition with active-record uniqueness constraints, like if a user double-submits the sign-up form.

Member

nilbus commented Jul 8, 2017

I would have made this a unique index, except that we actually have duplicate users in our production database. One example:

# select id, username, key, updated_at from users where key='[redacted]';
  id   |    key     |         updated_at
-------+------------+----------------------------
 14804 | [redacted] | 2014-06-05 11:59:17.267012
 14805 | [redacted] | 2014-06-05 11:59:13.151091

These could be due to a race condition with active-record uniqueness constraints, like if a user double-submits the sign-up form.

@kytrinyx

This comment has been minimized.

Show comment
Hide comment
@kytrinyx

kytrinyx Jul 8, 2017

Member

Oh my goodness, I can't believe I missed that. Thank you!

Member

kytrinyx commented Jul 8, 2017

Oh my goodness, I can't believe I missed that. Thank you!

@kytrinyx

This comment has been minimized.

Show comment
Hide comment
@kytrinyx

kytrinyx Jul 8, 2017

Member

This is saying 0 files changed. I'm seeing one commit with no changes. Would you take a look? (I can also make a migration to add the key, just curious what's going on here).

Member

kytrinyx commented Jul 8, 2017

This is saying 0 files changed. I'm seeing one commit with no changes. Would you take a look? (I can also make a migration to add the key, just curious what's going on here).

@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Jul 8, 2017

Member
Member

nilbus commented Jul 8, 2017

@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Jul 8, 2017

Member
Member

nilbus commented Jul 8, 2017

Index User lookup by key
Without this index, the DB must scan all 150k users for each user lookup
by key.
@nilbus

This comment has been minimized.

Show comment
Hide comment
@nilbus

nilbus Jul 8, 2017

Member

@kytrinyx All fixed. Sorry about that!

Member

nilbus commented Jul 8, 2017

@kytrinyx All fixed. Sorry about that!

@kytrinyx kytrinyx merged commit 36aedc5 into exercism:master Jul 8, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage remained the same at 86.524%
Details
@kytrinyx

This comment has been minimized.

Show comment
Hide comment
@kytrinyx

kytrinyx Jul 8, 2017

Member

Sweet, thanks!

Member

kytrinyx commented Jul 8, 2017

Sweet, thanks!

@nilbus nilbus deleted the nilbus:index_user_key branch Jul 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment