TypeError: catching classes that do not inherit from BaseException is not allowed #3614

muammar · 2020-03-21T06:31:39Z

I upgraded my python3 version on my MacBook pro to 3.7.7, and now distributed is not able to serialize/deserialize.

distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 101, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 193, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.core - ERROR - unpackb() got an unexpected keyword argument 'strict_map_key'
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/core.py", line 346, in handle_comm
    msg = await comm.read()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/comm/tcp.py", line 212, in read
    frames, deserialize=self.deserialize, deserializers=deserializers
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/comm/utils.py", line 75, in from_frames
    res = _from_frames()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/comm/utils.py", line 61, in _from_frames
    frames, deserialize=deserialize, deserializers=deserializers
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 101, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 193, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 101, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 193, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
Traceback (most recent call last):
  File "cu_training.py", line 51, in <module>
    cluster = LocalCluster()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/local.py", line 211, in __init__
    security=security,
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/spec.py", line 256, in __init__
    self.sync(self._start)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/cluster.py", line 161, in sync
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/utils.py", line 348, in sync
    raise exc.with_traceback(tb)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/utils.py", line 332, in f
    result[0] = yield future
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/spec.py", line 289, in _start
    await super()._start()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/cluster.py", line 60, in _start
    self.scheduler_info = await comm.read()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/comm/tcp.py", line 212, in read
    frames, deserialize=self.deserialize, deserializers=deserializers
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/comm/utils.py", line 75, in from_frames
    res = _from_frames()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/comm/utils.py", line 61, in _from_frames
    frames, deserialize=deserialize, deserializers=deserializers
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 101, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/protocol/core.py", line 193, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/utils.py", line 200, in ignoring
    yield
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/spec.py", line 607, in close_clusters
    cluster.close(timeout=10)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/cluster.py", line 82, in close
    return self.sync(self._close, callback_timeout=timeout)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/cluster.py", line 161, in sync
    return sync(self.loop, func, *args, **kwargs)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/utils.py", line 348, in sync
    raise exc.with_traceback(tb)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/utils.py", line 332, in f
    result[0] = yield future
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/spec.py", line 380, in _close
    self.scale(0)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/deploy/spec.py", line 444, in scale
    v["name"] for v in self.scheduler_info["workers"].values()
KeyError: 'workers'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/utils.py", line 201, in ignoring
    except exceptions as e:
TypeError: catching classes that do not inherit from BaseException is not allowed

Have any of you experienced the same situation?

The text was updated successfully, but these errors were encountered:

jrbourbeau · 2020-03-21T14:52:16Z

Thanks for raising an issue @muammar. This error: TypeError: unpackb() got an unexpected keyword argument 'strict_map_key' is coming from msgpack. What version of msgpack do you have installed? We recently bumped the minimum supported version to msgpack==0.6.0, which should have the 'strict_map_key' keyword argument

muammar · 2020-03-21T18:35:41Z

You are correct. That solved it. I should close the report.

jrbourbeau · 2020-03-21T19:59:44Z

Glad to hear that fixed things

rjurney · 2020-06-09T23:30:10Z

I have this bug.

distributed.worker - INFO -       Start worker at:      tcp://127.0.0.1:38591
distributed.worker - INFO -          Listening to:      tcp://127.0.0.1:38591
distributed.worker - INFO -          dashboard at:            127.0.0.1:38859
distributed.worker - INFO - Waiting to connect to:       tcp://127.0.0.1:8786
distributed.worker - INFO -       Start worker at:      tcp://127.0.0.1:40635
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -          Listening to:      tcp://127.0.0.1:40635
distributed.worker - INFO -          dashboard at:            127.0.0.1:45265
distributed.worker - INFO - Waiting to connect to:       tcp://127.0.0.1:8786
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          2
distributed.worker - INFO -                Memory:                    8.36 GB
distributed.worker - INFO -       Local Directory: /data/matcha.doodle.backend/dask-worker-space/worker-0p7c8zw4
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO -               Threads:                          2
distributed.worker - INFO -                Memory:                    8.36 GB
distributed.worker - INFO -       Local Directory: /data/matcha.doodle.backend/dask-worker-space/worker-ek57dqzs
distributed.worker - INFO - -------------------------------------------------
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 107, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 199, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.nanny - ERROR - Failed to start worker
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/nanny.py", line 713, in run
    await worker
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/worker.py", line 1060, in start
    await self._register_with_scheduler()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/worker.py", line 844, in _register_with_scheduler
    response = await future
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/comm/tcp.py", line 212, in read
    frames, deserialize=self.deserialize, deserializers=deserializers
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/comm/utils.py", line 75, in from_frames
    res = _from_frames()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/comm/utils.py", line 61, in _from_frames
    frames, deserialize=deserialize, deserializers=deserializers
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 107, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 199, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 107, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 199, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.nanny - ERROR - Failed to start worker
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/nanny.py", line 713, in run
    await worker
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/worker.py", line 1060, in start
    await self._register_with_scheduler()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/worker.py", line 844, in _register_with_scheduler
    response = await future
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/comm/tcp.py", line 212, in read
    frames, deserialize=self.deserialize, deserializers=deserializers
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/comm/utils.py", line 75, in from_frames
    res = _from_frames()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/comm/utils.py", line 61, in _from_frames
    frames, deserialize=deserialize, deserializers=deserializers
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 107, in loads
    msg = loads_msgpack(small_header, small_payload)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/protocol/core.py", line 199, in loads_msgpack
    return msgpack.loads(payload, use_list=False, **msgpack_opts)
  File "msgpack/_unpacker.pyx", line 161, in msgpack._unpacker.unpackb
TypeError: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.comm.tcp - WARNING - Closing dangling stream in <TCP  local=tcp://127.0.0.1:34912 remote=tcp://127.0.0.1:8786>
distributed.comm.tcp - WARNING - Closing dangling stream in <TCP  local=tcp://127.0.0.1:34914 remote=tcp://127.0.0.1:8786>
distributed.nanny - ERROR - Failed while trying to start worker process: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.nanny - ERROR - Failed while trying to start worker process: unpackb() got an unexpected keyword argument 'strict_map_key'
distributed.dask_worker - INFO - End worker
Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/bin/dask-worker", line 11, in <module>
    sys.exit(go())
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/cli/dask_worker.py", line 436, in go
    main()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/cli/dask_worker.py", line 422, in main
    loop.run_sync(run)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/tornado/ioloop.py", line 532, in run_sync
    return future_cell[0].result()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/cli/dask_worker.py", line 416, in run
    await asyncio.gather(*nannies)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/asyncio/tasks.py", line 537, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/nanny.py", line 251, in start
    response = await self.instantiate()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/nanny.py", line 334, in instantiate
    result = await self.process.start()
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/nanny.py", line 528, in start
    msg = await self._wait_until_connected(uid)
  File "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/distributed/nanny.py", line 643, in _wait_until_connected
    raise msg
TypeError: exceptions must derive from BaseException

mrocklin · 2020-06-10T13:47:56Z

The comment above was then raised in another issue. It turned out to be an installation issue with msgpack.

richardzhu · 2020-08-08T05:03:53Z

FYI: I had to do pip uninstall msgpack-python and pip install msgpack to get this to work for me, per the discussion on upgrading to 1.0 here.

jaymegordo · 2022-02-13T21:11:09Z

Having same issue here. My environment is managed with poetry, so I'm sure I could force some manual installs, but not ideal if something isn't figuring out the correct versions by default.

edit: I had an old package which still required msgpack-python instead of the newer msgpack which was causing the conflict.

jthetzel · 2023-07-07T13:55:48Z

An installation of intake had a dependency on msgpack-python version 0.5.6. So, we had to first

poetry remove intake

which removed msgpack-python.
Then we could

poetry add msgpack

to install a recent msgpack. Then add back intake:

poetry add intake

and both distributed and intake work correctly.

jrbourbeau closed this as completed Mar 21, 2020

stepan-tsirkin mentioned this issue May 12, 2022

problems with ray and msgpack wannier-berri/wannier-berri#174

Open

LiYangkai mentioned this issue Jun 12, 2022

requirements.txt 疑似依赖存在问题 cc004/pcrjjc2#93

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError: catching classes that do not inherit from BaseException is not allowed #3614

TypeError: catching classes that do not inherit from BaseException is not allowed #3614

muammar commented Mar 21, 2020

jrbourbeau commented Mar 21, 2020

muammar commented Mar 21, 2020

jrbourbeau commented Mar 21, 2020

rjurney commented Jun 9, 2020

mrocklin commented Jun 10, 2020

richardzhu commented Aug 8, 2020

jaymegordo commented Feb 13, 2022 •

edited

jthetzel commented Jul 7, 2023

TypeError: catching classes that do not inherit from BaseException is not allowed #3614

TypeError: catching classes that do not inherit from BaseException is not allowed #3614

Comments

muammar commented Mar 21, 2020

jrbourbeau commented Mar 21, 2020

muammar commented Mar 21, 2020

jrbourbeau commented Mar 21, 2020

rjurney commented Jun 9, 2020

mrocklin commented Jun 10, 2020

richardzhu commented Aug 8, 2020

jaymegordo commented Feb 13, 2022 • edited

jthetzel commented Jul 7, 2023

jaymegordo commented Feb 13, 2022 •

edited