You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every-time this function is called, a new connection to MongoDB is established. This results in large number of connections to MongoDB. This can result in the ephemeral port range on the server getting exhausted on busy saltmasters.
Other returner modules such as redis, https://github.com/saltstack/salt/blob/master/salt/returners/redis_return.py#L175 establish a connection only initially, then re-use that connection pool. It seems the pymongo returner could also use a connection pool via a similar method to reduce the number of connections that are made as pooling is supported by the module.
Setup
# master job cache and salt returns go here
master_job_cache: mongo
mongo.uri: "mongodb://USERNAME:PASSWORD@host1:27017,host2:27017,host3:27017/database?replicaSet=rs0&readPreference=nearest&journal=true&connectTimeoutMS=10000"
mongo.indexes: true
Steps to Reproduce Issue
Enable mongo returner module. Then attempt to run a salt command on many hosts and watch the connections to MongoDB (netstat -tunap). The number of connections keeps increasing proportional to the number minions being targeted. This does not occur when using other returners such as the redis returner.
Versions Report
root@saltmater:~# salt --versions-report
Salt Version:
Salt: 2019.2.1
Dependency Versions:
cffi: Not Installed
cherrypy: unknown
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
ioflo: Not Installed
Jinja2: 2.10
libgit2: 0.26.0
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: 0.26.2
Python: 3.6.8 (default, Jan 14 2019, 11:02:34)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
RAET: Not Installed
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist: Ubuntu 18.04 bionic
locale: UTF-8
machine: x86_64
release: 4.15.0-46-generic
system: Linux
version: Ubuntu 18.04 bionic
The text was updated successfully, but these errors were encountered:
Description of Issue
https://github.com/saltstack/salt/blob/master/salt/returners/mongo_future_return.py#L158
Every-time this function is called, a new connection to MongoDB is established. This results in large number of connections to MongoDB. This can result in the ephemeral port range on the server getting exhausted on busy saltmasters.
Other returner modules such as redis, https://github.com/saltstack/salt/blob/master/salt/returners/redis_return.py#L175 establish a connection only initially, then re-use that connection pool. It seems the pymongo returner could also use a connection pool via a similar method to reduce the number of connections that are made as pooling is supported by the module.
Setup
Steps to Reproduce Issue
Enable mongo returner module. Then attempt to run a salt command on many hosts and watch the connections to MongoDB (netstat -tunap). The number of connections keeps increasing proportional to the number minions being targeted. This does not occur when using other returners such as the redis returner.
Versions Report
The text was updated successfully, but these errors were encountered: