How to reduce Mitogen logs when enabled Ansible debug mode?

[YanFenGuo]

How to reduce Mitogen logs when enabled Ansible debug mode?

I added Mitogen in our Ansible Operator by following installation guide here: https://mitogen.networkgenomics.com/ansible_detailed.html#installation

It works fine, but the problem is when we enabled Ansible debug with -vvv, there are too many mitogen output in Ansible logs and make it is hard to analyze Ansible log. Below is an example with a smiple Ansible task which just output an variable. Is there a way to not output these Mitogen logs when debugging Ansible?

sh-4.4$ ansible-playbook test.yaml  -vvv  -e 'ansible_python_interpreter=/usr/bin/python3'
ansible-playbook 2.10.10
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/usr/share/ansible/openshift']
  ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.8.16 (default, May 31 2023, 12:44:21) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)]
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Parsed /etc/ansible/hosts inventory source with ini plugin
redirecting (type: callback) ansible.builtin.profile_tasks to ansible.posix.profile_tasks
redirecting (type: callback) ansible.builtin.timer to ansible.posix.timer
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: test.yaml *******************************************************************************************************************
1 plays in test.yaml

PLAY [localhost] **********************************************************************************************************************
[top  3750] 06:33:35.032286 D ansible_mitogen.affinity: CPU mask for Ansible top-level process: 0x000002
[top  3750] 06:33:35.032982 D ansible_mitogen.process: inherited open file limits: soft=1048576 hard=1048576
[top  3750] 06:33:35.033187 D ansible_mitogen.process: max open files already set to hard limit: 1048576
[mux  3764] 06:33:35.035466 D ansible_mitogen.affinity: CPU mask for MuxProcess 0: 0x000001
[mux  3764] 06:33:35.044364 D mitogen.service: Pool(f670, size=32, th='MainThread'): initialized
[mux  3764] 06:33:35.045119 D ansible_mitogen.process: Service pool configured: size=32

TASK [Gathering Facts] ****************************************************************************************************************
task path: /tmp/ansible-operator/runner/fncm.ibm.com/v1/FNCMCluster/ecm-fncm-s/fncmdeploy/artifacts/test.yaml:2
Tuesday 31 October 2023  06:33:35 +0000 (0:00:00.102)       0:00:00.102 ******* 
[task 3798] 06:33:35.110725 D ansible_mitogen.affinity: CPU mask for WorkerProcess: 0x000004
[task 3798] 06:33:35.118673 D ansible_mitogen.process: will use multiplexer 0 (/tmp/mitogen_unix_sb3y2wk5.sock) to connect to "localhost"
[task 3798] 06:33:35.118873 D mitogen.unix: client: connecting to /tmp/mitogen_unix_sb3y2wk5.sock
[task 3798] 06:33:35.119398 D mitogen.unix: client: local ID is 1, remote is 0
[mux  3764] 06:33:35.119523 D mitogen.unix: listener: accepted connection from PID 3798: unix_client.3798
[mux  3764] 06:33:35.120538 D mitogen.parent: creating connection to context 2 using mitogen.parent
[mux  3764] 06:33:35.430445 D mitogen.parent: command line for Connection(None): /usr/bin/python3 -c "import codecs,os,sys;_=codecs.decode;exec(_(_(\"eNqVkltPwkAQhd/5Fb7NbiywrUhtYxMJXhO5pEGJQUJ62epG2G22hYq/3ikYKPhgfGn22zkzc7I9vjH2VNZIRcoJrWmjqJBIThASpT8IdWvlOV6mFmGGyRjds29USWPV3HI0VxknfhV0FcZVKBC2C5cyWOD6CZt6HlwHuhASAhlXS9Z04lrTSzCdTSVbZw3+yaNlHoRzjl3NZaaboZDNdJ2/KwnukeJYYDVs+Pd6vLeYZe4crLjOhJIT96zsLEe62MPlSmi8heHL6H7Qnz12nvrd+5vr2a0/6M3Gfmc4vPFh6oEJm9fY6Tv+3TPDwqH1rQZxTg4LxiGeAlmIXL1x6eLfYraFH3YlwkU9kdGirlKug1zp+rkTtuwoajtO3dF5K3PP7HaLAq3hmkKLnBPTgN7DaIDtrxLQXqRifANa63ozUsYjxlkSQwE6BNrQPIiJaTPnghrwJVKclKTeXjc2oAihTEyS/izobs7bFBypi7/Uv12ahy53+bLoN0fL6bM=\".encode(),\"base64\"),\"zip\"))"
[mux  3764] 06:33:35.436582 D mitogen.parent: child for Connection(None) started: pid:3804 stdin:76 stdout:76 stderr:None
[mux  3764] 06:33:35.523779 D mitogen.parent: BootstrapProtocol(local.3804): first stage started succcessfully
[mux  3764] 06:33:35.525337 D mitogen.parent: BootstrapProtocol(local.3804): first stage received mitogen.core source
[mux  3764] 06:33:35.707068 D mitogen.parent: BootstrapProtocol(local.3804): new child booted successfully
[mux  3764] 06:33:35.707673 D mitogen.[local.3804]: Python version is 3.8.16 (default, May 31 2023, 12:44:21) 
[GCC 8.5.0 20210514 (Red Hat 8.5.0-18)]
[mux  3764] 06:33:35.707817 D mitogen.[local.3804]: Parent is context 0 (master); my ID is 2
[mux  3764] 06:33:35.707895 D mitogen.[local.3804]: pid:3804 ppid:3764 uid:1000720000/1000720000, gid:0/0 host:'ibm-fncm-operator-59b47cc699-9rt4s'
[mux  3764] 06:33:35.707982 D mitogen.[local.3804]: Recovered sys.executable: '/usr/bin/python3'
[mux  3764] 06:33:35.708837 D mitogen.parent: starting function call to local.3804: ansible_mitogen.target.init_child(log_level=10, candidate_temp_dirs=['/tmp/.ansible-${USER}/tmp', '/var/tmp', '/tmp'])
[mux  3764] 06:33:35.709842 D mitogen.responder: ansible is a package at /usr/local/lib/python3.8/site-packages/ansible/__init__.py with submodules ['_vendor', 'cli', 'collections', 'compat', 'config', 'constants', 'context', 'errors', 'executor', 'galaxy', 'inventory', 'module_utils', 'modules', 'parsing', 'playbook', 'plugins', 'release', 'template', 'utils', 'vars']
[mux  3764] 06:33:35.710286 D mitogen.responder: sending ansible (0.52 KiB) to local.3804
[mux  3764] 06:33:35.711035 D mitogen.responder: ansible.module_utils is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/__init__.py with submodules ['_text', 'ansible_release', 'api', 'basic', 'common', 'compat', 'connection', 'csharp', 'distro', 'facts', 'json_utils', 'parsing', 'powershell', 'pycompat24', 'service', 'six', 'splitter', 'urls', 'yumdnf']
[mux  3764] 06:33:35.711313 D mitogen.responder: sending ansible.module_utils (0.47 KiB) to local.3804
[mux  3764] 06:33:35.825446 D mitogen: PkgutilMethod(): <_frozen_importlib_external.SourceFileLoader object at 0x7f489e4ec880>.get_file_name('ansible.module_utils.distro') failed: ImportError('loader for ansible.module_utils.distro._distro cannot handle ansible.module_utils.distro')
[mux  3764] 06:33:35.825632 D mitogen: SysModulesMethod(): <module 'ansible.module_utils.distro._distro' from '/usr/local/lib/python3.8/site-packages/ansible/module_utils/distro/_distro.py'>.__name__ is 'ansible.module_utils.distro._distro', aborting
[mux  3764] 06:33:35.825774 D mitogen: ParentEnumerationMethod(): 'ansible.module_utils.distro' is PKG_DIRECTORY: '/usr/local/lib/python3.8/site-packages/ansible/module_utils/distro/__init__.py'
[mux  3764] 06:33:35.936695 D mitogen: PkgutilMethod(): <class '_frozen_importlib.FrozenImporter'>.get_file_name('zipimport') failed: AttributeError("type object 'FrozenImporter' has no attribute 'get_filename'")
[mux  3764] 06:33:35.936920 D mitogen: SysModulesMethod(): <module 'zipimport' (frozen)> has no __file__, aborting
[mux  3764] 06:33:35.937067 D mitogen: get_module_source('zipimport'): cannot find source
[mux  3764] 06:33:36.035155 D mitogen: PkgutilMethod(): path=None, source=None, aborting
[mux  3764] 06:33:36.035399 D mitogen: SysModulesMethod(): <module 'markupsafe._speedups' from '/usr/local/lib64/python3.8/site-packages/markupsafe/_speedups.cpython-38-x86_64-linux-gnu.so'>.__file__ is None, aborting
[mux  3764] 06:33:36.035656 D mitogen: get_module_source('markupsafe._speedups'): cannot find source
[mux  3764] 06:33:36.136676 D mitogen: PkgutilMethod(): path=None, source=None, aborting
[mux  3764] 06:33:36.136876 D mitogen: SysModulesMethod(): <module 'yaml._yaml' from '/usr/local/lib64/python3.8/site-packages/yaml/_yaml.cpython-38-x86_64-linux-gnu.so'>.__file__ is None, aborting
[mux  3764] 06:33:36.137077 D mitogen: get_module_source('yaml._yaml'): cannot find source
[mux  3764] 06:33:36.137530 D mitogen.responder: ansible.config is a package at /usr/local/lib/python3.8/site-packages/ansible/config/__init__.py with submodules ['data', 'manager']
[mux  3764] 06:33:36.137687 D mitogen.responder: sending ansible.config (0.22 KiB) to local.3804
[mux  3764] 06:33:36.137950 D mitogen.responder: sending ansible.config.data (0.90 KiB) to local.3804
[mux  3764] 06:33:36.140870 D mitogen.responder: sending ansible.config.manager (10.50 KiB) to local.3804
[mux  3764] 06:33:36.141674 D mitogen.responder: sending ansible.constants (5.65 KiB) to local.3804
[mux  3764] 06:33:36.141951 D mitogen.responder: sending ansible.context (2.00 KiB) to local.3804
[mux  3764] 06:33:36.142566 D mitogen.responder: ansible.errors is a package at /usr/local/lib/python3.8/site-packages/ansible/errors/__init__.py with submodules ['yaml_strings']
[mux  3764] 06:33:36.143317 D mitogen.responder: sending ansible.errors (6.55 KiB) to local.3804
[mux  3764] 06:33:36.143659 D mitogen.responder: sending ansible.errors.yaml_strings (2.91 KiB) to local.3804
[mux  3764] 06:33:36.143847 D mitogen.responder: sending ansible.module_utils._text (0.92 KiB) to local.3804
[mux  3764] 06:33:36.144209 D mitogen.responder: ansible.module_utils.common is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/common/__init__.py with submodules ['_collections_compat', '_json_compat', '_utils', 'collections', 'dict_transformations', 'file', 'json', 'network', 'parameters', 'process', 'removed', 'sys_info', 'text', 'validation', 'warnings']
[mux  3764] 06:33:36.144343 D mitogen.responder: sending ansible.module_utils.common (0.49 KiB) to local.3804
[mux  3764] 06:33:36.144542 D mitogen.responder: sending ansible.module_utils.common._collections_compat (1.03 KiB) to local.3804
[mux  3764] 06:33:36.144731 D mitogen.responder: sending ansible.module_utils.common._json_compat (0.81 KiB) to local.3804
[mux  3764] 06:33:36.144951 D mitogen.responder: sending ansible.module_utils.common._utils (1.32 KiB) to local.3804
[mux  3764] 06:33:36.145285 D mitogen.responder: sending ansible.module_utils.common.collections (2.44 KiB) to local.3804
[mux  3764] 06:33:36.145778 D mitogen.responder: sending ansible.module_utils.common.file (3.87 KiB) to local.3804
[mux  3764] 06:33:36.146488 D mitogen.responder: sending ansible.module_utils.common.parameters (4.43 KiB) to local.3804
[mux  3764] 06:33:36.146732 D mitogen.responder: sending ansible.module_utils.common.process (1.62 KiB) to local.3804
[mux  3764] 06:33:36.147204 D mitogen.responder: sending ansible.module_utils.common.sys_info (3.09 KiB) to local.3804
[mux  3764] 06:33:36.147409 D mitogen.responder: ansible.module_utils.common.text is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/common/text/__init__.py with submodules ['converters', 'formatters']
[mux  3764] 06:33:36.147528 D mitogen.responder: sending ansible.module_utils.common.text (0.33 KiB) to local.3804
[mux  3764] 06:33:36.206733 D mitogen.responder: sending ansible.module_utils.common.text.converters (5.15 KiB) to local.3804
[mux  3764] 06:33:36.207402 D mitogen.responder: sending ansible.module_utils.common.text.formatters (2.61 KiB) to local.3804
[mux  3764] 06:33:36.209221 D mitogen.responder: sending ansible.module_utils.common.validation (6.61 KiB) to local.3804
[mux  3764] 06:33:36.209572 D mitogen.responder: sending ansible.module_utils.common.warnings (1.19 KiB) to local.3804
[mux  3764] 06:33:36.209859 D mitogen.responder: ansible.module_utils.compat is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/compat/__init__.py with submodules ['_selectors2', 'importlib', 'paramiko', 'selectors']
[mux  3764] 06:33:36.209980 D mitogen.responder: sending ansible.module_utils.compat (0.31 KiB) to local.3804
[mux  3764] 06:33:36.210186 D mitogen.responder: ansible.module_utils.distro is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/distro/__init__.py with submodules ['_distro']
[mux  3764] 06:33:36.210379 D mitogen.responder: sending ansible.module_utils.distro (1.59 KiB) to local.3804
[mux  3764] 06:33:36.215760 D mitogen.responder: sending ansible.module_utils.distro._distro (15.88 KiB) to local.3804
[mux  3764] 06:33:36.216008 D mitogen.responder: ansible.module_utils.parsing is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/parsing/__init__.py with submodules ['convert_bool']
[mux  3764] 06:33:36.216141 D mitogen.responder: sending ansible.module_utils.parsing (0.27 KiB) to local.3804
[mux  3764] 06:33:36.216419 D mitogen.responder: sending ansible.module_utils.parsing.convert_bool (1.32 KiB) to local.3804
[mux  3764] 06:33:36.216879 D mitogen.responder: sending ansible.module_utils.pycompat24 (2.97 KiB) to local.3804
[mux  3764] 06:33:36.218386 D mitogen.responder: ansible.module_utils.six is a package at /usr/local/lib/python3.8/site-packages/ansible/module_utils/six/__init__.py with submodules []
[mux  3764] 06:33:36.220680 D mitogen.responder: sending ansible.module_utils.six (12.14 KiB) to local.3804
[mux  3764] 06:33:36.221077 D mitogen.responder: ansible.parsing is a package at /usr/local/lib/python3.8/site-packages/ansible/parsing/__init__.py with submodules ['ajson', 'dataloader', 'mod_args', 'plugin_docs', 'quoting', 'splitter', 'utils', 'vault', 'yaml']
[mux  3764] 06:33:36.221277 D mitogen.responder: sending ansible.parsing (1.06 KiB) to local.3804
[mux  3764] 06:33:36.221560 D mitogen.responder: sending ansible.parsing.quoting (1.17 KiB) to local.3804
[mux  3764] 06:33:36.221872 D mitogen.responder: ansible.parsing.yaml is a package at /usr/local/lib/python3.8/site-packages/ansible/parsing/yaml/__init__.py with submodules ['constructor', 'dumper', 'loader', 'objects']
[mux  3764] 06:33:36.222067 D mitogen.responder: sending ansible.parsing.yaml (1.02 KiB) to local.3804
[mux  3764] 06:33:36.222836 D mitogen.responder: sending ansible.parsing.yaml.objects (4.84 KiB) to local.3804
[mux  3764] 06:33:36.223334 D mitogen.responder: ansible.utils is a package at /usr/local/lib/python3.8/site-packages/ansible/utils/__init__.py with submodules ['cmd_functions', 'collection_loader', 'color', 'context_objects', 'display', 'encrypt', 'fqcn', 'galaxy', 'hashing', 'helpers', 'jsonrpc', 'listify', 'multiprocessing', 'path', 'plugin_docs', 'py3compat', 'sentinel', 'shlex', 'singleton', 'ssh_functions', 'unicode', 'unsafe_proxy', 'vars', 'version']
[mux  3764] 06:33:36.223489 D mitogen.responder: sending ansible.utils (1.29 KiB) to local.3804
[mux  3764] 06:33:36.224107 D mitogen.responder: sending ansible.utils.color (3.82 KiB) to local.3804
[mux  3764] 06:33:36.224444 D mitogen.responder: sending ansible.utils.context_objects (2.21 KiB) to local.3804
[mux  3764] 06:33:36.226108 D mitogen.responder: sending ansible.utils.display (8.37 KiB) to local.3804
[mux  3764] 06:33:36.226346 D mitogen.responder: sending ansible.utils.fqcn (1.19 KiB) to local.3804
[mux  3764] 06:33:36.226868 D mitogen.responder: sending ansible.utils.path (3.81 KiB) to local.3804
[mux  3764] 06:33:36.227175 D mitogen.responder: sending ansible.utils.py3compat (1.87 KiB) to local.3804
[mux  3764] 06:33:36.227460 D mitogen.responder: sending ansible.utils.singleton (0.91 KiB) to local.3804
[mux  3764] 06:33:36.227865 D mitogen.responder: sending ansible.utils.unsafe_proxy (3.80 KiB) to local.3804
[mux  3764] 06:33:36.228142 D mitogen.responder: sending ansible.module_utils.basic (42.78 KiB) to local.3804
[mux  3764] 06:33:36.229078 D mitogen.responder: sending ansible.module_utils.json_utils (2.68 KiB) to local.3804
[mux  3764] 06:33:36.229542 D mitogen.responder: sending ansible.release (1.00 KiB) to local.3804
[mux  3764] 06:33:36.230031 D mitogen.responder: ansible_mitogen is a package at /opt/ansible/mitogen-0.3.4/ansible_mitogen/__init__.py with submodules ['affinity', 'compat', 'connection', 'loaders', 'logging', 'mixins', 'module_finder', 'parsing', 'planner', 'plugins', 'process', 'runner', 'services', 'strategy', 'target', 'transport_config', 'utils']
[mux  3764] 06:33:36.230170 D mitogen.responder: sending ansible_mitogen (0.42 KiB) to local.3804
[mux  3764] 06:33:36.519385 D mitogen.responder: sending ansible_mitogen.target (7.94 KiB) to local.3804
[mux  3764] 06:33:36.728483 D mitogen.responder: sending mitogen.parent (18.84 KiB) to local.3804
[mux  3764] 06:33:36.735578 D mitogen.responder: sending mitogen.select (1.60 KiB) to local.3804
[mux  3764] 06:33:36.840029 D mitogen.responder: sending mitogen.service (8.88 KiB) to local.3804
[mux  3764] 06:33:36.840312 D mitogen.responder: sending ansible_mitogen.runner (8.75 KiB) to local.3804
[mux  3764] 06:33:36.849042 D mitogen.responder: sending mitogen.fork (2.61 KiB) to local.3804
[mux  3764] 06:33:36.849692 D mitogen.importer.[local.3804]: received ansible
[mux  3764] 06:33:36.849815 D mitogen.importer.[local.3804]: received ansible.module_utils
[mux  3764] 06:33:36.849890 D mitogen.importer.[local.3804]: received ansible.config
[mux  3764] 06:33:36.849948 D mitogen.importer.[local.3804]: received ansible.config.data
[mux  3764] 06:33:36.850002 D mitogen.importer.[local.3804]: received ansible.config.manager
[mux  3764] 06:33:36.850055 D mitogen.importer.[local.3804]: received ansible.constants
[mux  3764] 06:33:36.850110 D mitogen.importer.[local.3804]: received ansible.context
[mux  3764] 06:33:36.850160 D mitogen.importer.[local.3804]: received ansible.errors
.......
 

[YanFenGuo]

I fixed for our project by modifying Mitogen log and always only output error log, so I'm closing this as I don't need a fix from Mitogen anymore.

sed -i 's/logging.DEBUG/logging\.ERROR/g' /opt/ansible/mitogen-0.3.4/ansible_mitogen/logging.py