Queue and Variable assume the scheduler address is identical across workers

[lr4d]

What happened:
When using a distributed cluster wherein different workers may have a different scheduler address (because of proxying / network configuration), one runs into errors when attempting to deserialize a distributed.Queue or Variable as these data structures include an attribute which indicates the scheduler address on the worker on which it was created.

See

distributed/distributed/queues.py

Line 281 in 6ee7f89

def __setstate__(self, state):

,

distributed/distributed/variable.py

Line 230 in 1d7640b

def __setstate__(self, state):

.

As the scheduler address may be different on the worker on which it is de-serialized, the de-serialization would leave the object in an unusable state.
Traceback:

distributed.worker - WARNING - Could not deserialize task Traceback (most recent call last): File "/workspace/venv/lib/python3.6/site-packages/distributed/worker.py", line 2441, in _maybe_deserialize_task function, args, kwargs = _deserialize(*self.tasks[key]) File "/workspace/venv/lib/python3.6/site-packages/distributed/worker.py", line 3277, in _deserialize args = pickle.loads(args) File "/workspace/venv/lib/python3.6/site-packages/distributed/protocol/pickle.py", line 66, in loads return pickle.loads(x) File "/workspace/venv/lib/python3.6/site-packages/distributed/variable.py", line 233, in __setstate__ client = get_client(address) File "/workspace/venv/lib/python3.6/site-packages/distributed/worker.py", line 3129, in get_client return Client(address, timeout=timeout) File "/workspace/venv/lib/python3.6/site-packages/distributed/client.py", line 674, in __init__ self.connection_args = self.security.get_connection_args("client") File "/workspace/venv/lib/python3.6/site-packages/distributed/security.py", line 222, in get_connection_args "ssl_context": self._get_tls_context(tls, ssl.Purpose.SERVER_AUTH), File "/workspace/venv/lib/python3.6/site-packages/distributed/security.py", line 201, in _get_tls_context ctx.load_cert_chain(cert_path, key_path) FileNotFoundError: [Errno 2] No such file or directory

What you expected to happen:
Contrast with

distributed/distributed/semaphore.py

Line 502 in 6ee7f89

def __setstate__(self, state):

.

For the record, this came up in an internal discussion involving @jonashaag , @crepererum and @fjetter . Credit goes to them

[mrocklin]

So we could drop the address from the state. I suspect that it was originally there in order to verify that users weren't passing queues/variables between clusters. This is an infrequent problem though, so maybe we can skip it.

[fjetter]

For reference, the lock does it also "right" by asking the worker for an active client, see

distributed/distributed/lock.py

Line 172 in 1d7640b

def __reduce__(self):

[mrocklin]

If you all want to resolve this, that would be welcome