Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse DB connections in Main thread #31

Closed
timvaillancourt opened this issue Jul 10, 2016 · 0 comments · Fixed by #33
Closed

Reuse DB connections in Main thread #31

timvaillancourt opened this issue Jul 10, 2016 · 0 comments · Fixed by #33

Comments

@timvaillancourt
Copy link
Contributor

timvaillancourt commented Jul 10, 2016

In version 0.2.0 we do not always reuse Mongo DB connections. DB connections cannot be reused across threads (must be reopened in each thread), but there are many connections in the "MainProcess" thread that could be reused by passing a DB class from Backup.py to ReplsetHandler.py + ShardingHandler.py.

Here is the DB:connect debug output for a 2-shard cluster (11 x DB connections created!!!):

[tim@centos7 mongodb_consistent_backup]$ grep 'DB:connect' log 
[2016-07-08 19:41:40,493] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongos1:27017
[2016-07-08 19:41:40,500] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongos1:27017
[2016-07-08 19:41:50,602] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongos1:27017
[2016-07-08 19:41:50,612] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongod1:27017
[2016-07-08 19:41:50,626] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongod3:27017
[2016-07-08 19:41:50,836] [DEBUG] [OplogTail-1] [DB:connect:23] Getting MongoDB connection to centos7-mongod2:27017
[2016-07-08 19:41:50,839] [DEBUG] [OplogTail-2] [DB:connect:23] Getting MongoDB connection to centos7-mongod4:27017
[2016-07-08 19:41:50,856] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongos1:27017
[2016-07-08 19:41:50,884] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongos1:27017
[2016-07-08 19:41:50,890] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongod1:27017
[2016-07-08 19:41:50,905] [DEBUG] [MainProcess] [DB:connect:23] Getting MongoDB connection to centos7-mongod3:27017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant