UnicodeDecodeError at ProcessCollector.collect()

[bnopne]

Hi, i get the following error when i try to run my app with Gunicorn:

Failed to read config file: gunicornconfig.py
Traceback (most recent call last):
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/gunicorn/app/base.py", line 93, in get_config_from_filename
    execfile_(filename, cfg, cfg)
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/gunicorn/_compat.py", line 72, in execfile_
    return six.exec_(code, *args)
  File "gunicornconfig.py", line 1, in <module>
    from prometheus_client import multiprocess
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/prometheus_client/__init__.py", line 5, in <module>
    from . import process_collector
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/prometheus_client/process_collector.py", line 93, in <module>
    PROCESS_COLLECTOR = ProcessCollector()
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/prometheus_client/process_collector.py", line 39, in __init__
    registry.register(self)
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/prometheus_client/core.py", line 50, in register
    names = self._get_names(collector)
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/prometheus_client/core.py", line 86, in _get_names
    for metric in desc_func():
  File "/home/vasilek/coller_app/env/lib/python3.6/site-packages/prometheus_client/process_collector.py", line 56, in collect
    parts = (stat.read().split(')')[-1].split())
  File "/home/vasilek/.pyenv/versions/3.6.4/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 21: invalid continuation byte

App is started with command: gunicorn -b 0.0.0.0 app:api --config gunicornconfig.py --workers=4 --log-level=DEBUG --worker-class=aiohttp.worker.GunicornWebWorker --reload

Contents of gunicornconfig.py are taken straight from multiprocessing tutorial for this module:

from prometheus_client import multiprocess


def child_exit(server, worker):
    multiprocess.mark_process_dead(worker.pid)

Other packages:

python 3.6.4
gunicorn 19.7.1
prometheus-client 0.1.0

[brian-brazil]

I'm guessing you have some non-ASCII process names, and we should be opening the stat file in binary mode.

[bnopne]

@brian-brazil
I changed the following lines in process_collector.py (lines 56-57):

with open(os.path.join(pid, 'stat')) as stat:
    parts = (stat.read().split(')')[-1].split())

to:

with open(os.path.join(pid, 'stat'), 'rb') as stat:
    print(stat.read())

Got this output:

b'9832 (/home/vasilek/\xd0) R 9828 9828 5755 34818 9828 4194304 4342 0 0 0 36 2 0 0 20 0 1 0 355850 102895616 5768 18446744073709551615 4194304 6707556 140729189621072 0 0 0 0 16781312 2 0 0 0 17 1 0 0 0 0 0 8805840 9211976 20926464 140729189629641 140729189629956 140729189629956 140729189633948 0\n'

[brian-brazil]

Yip, that's not ASCII. Could you send a pull request?

[bnopne]

Sure, will do in a couple of days.