make exceptions picklable

[thehesiod]

No description provided.

[jamesls]

Out of curiosity, how are you using pickled exceptions? I'd like to get a better understanding of how botocore exceptions are being used.

[thehesiod]

yep! I've created a S3 process pool to be able to handle > 1000 requests/s. To do this I need to pickle the exceptions across the process boundary.

[t0m]

Another use case for pickling exceptions is multiprocessing. I have a script that spawns a number of processes to handle SQS messages, and if a ClientError is raised I get the following error instead of the real one:

Exception in thread Thread-8:
Traceback (most recent call last):
  File "/Users/tterrace/.pyenv/versions/3.5.1/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/Users/tterrace/.pyenv/versions/3.5.1/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/tterrace/.pyenv/versions/3.5.1/lib/python3.5/multiprocessing/pool.py", line 429, in _handle_results
    task = get()
  File "/Users/tterrace/.pyenv/versions/3.5.1/lib/python3.5/multiprocessing/connection.py", line 251, in recv
    return ForkingPickler.loads(buf.getbuffer())
TypeError: __init__() missing 1 required positional argument: 'operation_name'