Log worker info on error

[j-bennet]

Different version of #7847.

• Tests added / passed
• Passes pre-commit run --all-files

How this works:

import distributed

class BadNumbers(Exception):
    def __str__(self):
        return "I didn't like these numbers: " + " ".join(f"{x:.1f}" for x in self.args)

def f():
    raise BadNumbers(1, 2.5, 3)

client = distributed.Client()
client.submit(f).result()

2023-07-20 15:02:50,741 - distributed.worker - WARNING - Compute Failed
Key:       f-b49acabd688e43ee290e83e090a617ea
Function:  f
args:      ()
kwargs:    {}
Exception: 'BadNumbers(1, 2.5, 3)'

2023-07-20 15:02:50,743 - distributed.client - ERROR - Compute Failed
Key:       f-b49acabd688e43ee290e83e090a617ea
Function:  f
args:      ()
kwargs:    {}
Exception: BadNumbers(1, 2.5, 3)
Worker:    {'tcp://127.0.0.1:63910'}
---------------------------------------------------------------------------
BadNumbers                                Traceback (most recent call last)
Cell In[3], line 1
----> 1 client.submit(f).result()

Cell In[2], line 8, in f()
      7 def f():
----> 8     raise BadNumbers(1, 2.5, 3)

BadNumbers: I didn't like these numbers: 1.0 2.5 3.0

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       20 files  ±0         20 suites  ±0   11h 23m 50s ⏱️ - 41m 13s
  3 754 tests ±0    3 638 ✔️  -   5     106 💤 ±0  10 ❌ +  6 
36 314 runs  ±0  34 521 ✔️  - 36  1 746 💤  - 2  47 ❌ +39 

For more details on these failures, see this check.

Results for commit 23ce7b3. ± Comparison against base commit 4c4748a.

♻️ This comment has been updated with latest results.

[crusaderky]

Instead of:

2023-06-23 15:37:47,237 - distributed.worker - WARNING - Compute Failed
Key:       f-78eea523b6ed747be234c3bfbc24c116
Function:  f
args:      ()
kwargs:    {}
Exception: 'BadNumbers(1, 2.5, 3)'

2023-06-23 15:37:47,238 - distributed.client - ERROR - in task: f-78eea523b6ed747be234c3bfbc24c116
on worker:{'tcp://127.0.0.1:54346'}

Wouldn't it be much cleaner and simpler to just add a line to the already existing message?

2023-06-23 15:37:47,237 - distributed.worker - WARNING - Compute Failed
Key:       f-78eea523b6ed747be234c3bfbc24c116
Function:  f
args:      ()
kwargs:    {}
Exception: 'BadNumbers(1, 2.5, 3)'
Worker:    tcp://127.0.0.1:54346

[j-bennet]

@crusaderky

Wouldn't it be much cleaner and simpler to just add a line to the already existing message?

It's not so easy, unfortunately. I wish every Python exception had something like a read-write message property that you could easily modify, and we would not need to jump through all these hoops, but that's not the case.

This PR is based on your suggestion here:

#7847 (comment)

[j-bennet]

@crusaderky I made changes as we discussed. The message emitted on client is now very similar to the one emitted on worker.

[crusaderky]

.