Multiprocess application sometimes throwing UnicodeDecodeError in prometheus_client/mmap_dict.py

[Velan987]

Hi
I have multiprocess enabled application with flask exporter, sometimes it is throwing below error
File "usr/local/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 44, in _read_all_values
yield encoded_key.decode('utf-8'), value, pos
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 18: invalid continuation byte.

After this error my application is crashed. Can someone please help me on this?

[csmarchbanks]

Hmm, I haven't seen this one before, is this something that can be reproduced? Some version and environment information may also be helpful.

[Velan987]

Hi @csmarchbanks, we are using prometheus-client 0.15.0 in docker environment
Below is the full error:

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2051, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in full_dispatch_request
return self.finalize_request(rv)
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1523, in finalize_request
response = self.process_response(response)
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1865, in process_response
response = self.ensure_sync(handler)(response)
File "/usr/local/lib/python3.8/site-packages/prometheus_flask_exporter/init.py", line 482, in after_request
request_duration_metric.labels(**request_duration_labels).observe(total_time)
File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 190, in labels
self._metrics[labelvalues] = self.class(
File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 558, in init
super().init(
File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 138, in init
self._metric_init()
File "/usr/local/lib/python3.8/site-packages/prometheus_client/metrics.py", line 586, in _metric_init
self._sum = values.ValueClass(self._type, self._name, self._name + '_sum', self._labelnames, self._labelvalues)
File "/usr/local/lib/python3.8/site-packages/prometheus_client/values.py", line 68, in init
self.__reset()
File "/usr/local/lib/python3.8/site-packages/prometheus_client/values.py", line 82, in __reset
files[file_prefix] = MmapedDict(filename)
File "/usr/local/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 78, in init
for key, _, pos in self._read_all_values():
File "/usr/local/lib/python3.8/site-packages/prometheus_client/mmap_dict.py", line 44, in _read_all_values
yield encoded_key.decode('utf-8'), value, pos
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 18: invalid continuation byte

[Velan987]

I have configured my application like below
metrics = MultiprocessInternalPrometheusMetrics(app, path="/api/metrics")
values.ValueClass = values.MultiProcessValue(os.getppid)

If I do not add "values.ValueClass = values.MultiProcessValue(os.getppid)" facing slowness in metrics endpoint