Lock wait timeout exceeded while running UPDATE on the users table

[c-hoeger]

Issue Description

We're seeing more and more errors like this:

I, [2019-03-11T14:18:44.001015 #26413]  INFO -- : [42194045-d8c6-4fb0-ad68-c6adc9dffeb8] [26413:239.97] Processing by BuildController#result as XML
I, [2019-03-11T14:18:44.001205 #26413]  INFO -- : [42194045-d8c6-4fb0-ad68-c6adc9dffeb8] [26413:239.97]   Parameters: {"project"=>"backend-samples-release-7.10.2"}
I, [2019-03-11T14:18:53.919283 #3429]  INFO -- : [d0ea0d61-2442-4e84-8e5a-de824b59d1a5] [3429:15755.68] Completed 500 Internal Server Error in 51911ms (ActiveRecord: 5
1414.0ms | Backend: 0.0ms | XML: 0.0ms)
F, [2019-03-11T14:18:53.927402 #3429] FATAL -- : [d0ea0d61-2442-4e84-8e5a-de824b59d1a5] [3429:15755.69]
F, [2019-03-11T14:18:53.927522 #3429] FATAL -- : [d0ea0d61-2442-4e84-8e5a-de824b59d1a5] [3429:15755.69] ActiveRecord::StatementInvalid (Mysql2::Error: Lock wait timeout exceeded; try restarting transaction: UPDATE `users` SET `updated_at` = '2019-03-11 13:18:02.503980', `last_logged_in_at` = '2019-03-11 13:18:02' WHERE `users`.`id` = 3):
F, [2019-03-11T14:18:53.927560 #3429] FATAL -- : [d0ea0d61-2442-4e84-8e5a-de824b59d1a5] [3429:15755.69]
F, [2019-03-11T14:18:53.927605 #3429] FATAL -- : [d0ea0d61-2442-4e84-8e5a-de824b59d1a5] [3429:15755.69] app/models/user.rb:858:in `mark_login!'
[d0ea0d61-2442-4e84-8e5a-de824b59d1a5] app/models/user.rb:186:in `find_with_credentials_via_ldap'
[d0ea0d61-2442-4e84-8e5a-de824b59d1a5] app/models/user.rb:146:in `find_with_credentials'
[d0ea0d61-2442-4e84-8e5a-de824b59d1a5] lib/authenticator.rb:48:in `extract_user'
[d0ea0d61-2442-4e84-8e5a-de824b59d1a5] app/controllers/application_controller.rb:30:in `extract_user'
[d0ea0d61-2442-4e84-8e5a-de824b59d1a5] config/initializers/wrap_parameters.rb:38:in `call'
I, [2019-03-11T14:18:55.925401 #6830]  INFO -- : [f9fa38f5-4c68-4bbc-82c3-0d220d822add] [6830:4979.94] Completed 500 Internal Server Error in 52166ms (ActiveRecord: 51725.1ms | Backend: 0.0ms | XML: 0.0ms)
F, [2019-03-11T14:18:55.937998 #6830] FATAL -- : [f9fa38f5-4c68-4bbc-82c3-0d220d822add] [6830:4979.95]
F, [2019-03-11T14:18:55.938128 #6830] FATAL -- : [f9fa38f5-4c68-4bbc-82c3-0d220d822add] [6830:4979.95] ActiveRecord::StatementInvalid (Mysql2::Error: Lock wait timeout exceeded; try restarting transaction: UPDATE `users` SET `updated_at` = '2019-03-11 13:18:04.195242', `last_logged_in_at` = '2019-03-11 13:18:04' WHERE `users`.`id` = 3):
F, [2019-03-11T14:18:55.938190 #6830] FATAL -- : [f9fa38f5-4c68-4bbc-82c3-0d220d822add] [6830:4979.95]
F, [2019-03-11T14:18:55.938347 #6830] FATAL -- : [f9fa38f5-4c68-4bbc-82c3-0d220d822add] [6830:4979.95] app/models/user.rb:858:in `mark_login!'
[f9fa38f5-4c68-4bbc-82c3-0d220d822add] app/models/user.rb:186:in `find_with_credentials_via_ldap'
[f9fa38f5-4c68-4bbc-82c3-0d220d822add] app/models/user.rb:146:in `find_with_credentials'
[f9fa38f5-4c68-4bbc-82c3-0d220d822add] lib/authenticator.rb:48:in `extract_user'
[f9fa38f5-4c68-4bbc-82c3-0d220d822add] app/controllers/application_controller.rb:30:in `extract_user'
[f9fa38f5-4c68-4bbc-82c3-0d220d822add] config/initializers/wrap_parameters.rb:38:in `call'

I assume that this is related to the ldap authentication, since I've seen a similar problem already before, see #3743

I guess that is also related to the fact, that we're mainly using the same single obs login in parallel
from many instances accessing our internal obs.