Fix handling of test-backend failures with subTest in parallel mode

[timabbott]

Apparently the Django subTest feature that we're using for a few of our test suites does not work well with Django's parallel test environment (or maybe with something else we're doing). The reproducer is the following:

diff --git a/zerver/tests/test_users.py b/zerver/tests/test_users.py
index 688a361776..ff152e36cb 100644
--- a/zerver/tests/test_users.py
+++ b/zerver/tests/test_users.py
@@ -428,6 +428,7 @@ class PermissionTest(ZulipTestCase):
         for field_dict in cordelia.profile_data:
             with self.subTest(field_name=field_dict['name']):
                 self.assertEqual(field_dict['value'], fields[field_dict['name']])  # type: ignore # Reason in comment
+                self.assertEqual(True, False)
             # Invalid index type for dict key, it must be str but field_dict values can be anything
 
         # Test admin user cannot set invalid profile data

The error output should be this:

FAIL: test_admin_user_can_change_profile_data (zerver.tests.test_users.PermissionTest) (field_name='Phone number')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tabbott/zulip/zerver/tests/test_users.py", line 431, in test_admin_user_can_change_profile_data
    self.assertEqual(True, False)
AssertionError: True != False

But you instead get this if you run a full test-backend:

Unfortunately, the subtest that failed cannot be pickled, so the parallel
test runner cannot handle it cleanly. Here is the pickling error:

> Can't pickle local object 'convert_exception_to_response.<locals>.inner'

You should re-run this test with --parallel=1 to reproduce the failure
with a cleaner failure message.

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/tabbott/zulip/zerver/lib/test_runner.py", line 209, in run_subsuite
    result = runner.run(deserialize_suite(subsuite))
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/runner.py", line 251, in run
    test(result)
  File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/home/tabbott/zulip/zerver/lib/test_runner.py", line 353, in run
    failed = run_test(test, result)
  File "/home/tabbott/zulip/zerver/lib/test_runner.py", line 130, in run_test
    test(result)  # unittest will handle skipping, error, failure and success.
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/testcases.py", line 213, in __call__
    super(SimpleTestCase, self).__call__(result)
  File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/case.py", line 613, in run
    self._feedErrorsToResult(result, outcome.errors)
  File "/usr/lib/python3.6/unittest/case.py", line 538, in _feedErrorsToResult
    result.addSubTest(test.test_case, test, exc_info)
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/runner.py", line 208, in addSubTest
    self.check_subtest_picklable(test, subtest)
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/runner.py", line 166, in check_subtest_picklable
    self._confirm_picklable(subtest)
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/runner.py", line 99, in _confirm_picklable
    pickle.loads(pickle.dumps(obj))
AttributeError: Can't pickle local object 'convert_exception_to_response.<locals>.inner'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tabbott/zulip/tools/test-backend", line 473, in <module>
    main()
  File "/home/tabbott/zulip/tools/test-backend", line 395, in main
    include_webhooks=include_webhooks)
  File "/home/tabbott/zulip/zerver/lib/test_runner.py", line 524, in run_tests
    result = self.run_suite(suite)
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/runner.py", line 567, in run_suite
    return runner.run(suite)
  File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
    test(result)
  File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/srv/zulip-py3-venv/lib/python3.6/site-packages/django/test/runner.py", line 370, in run
    subsuite_index, events = test_results.next(timeout=0.1)
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 761, in next
    raise value
AttributeError: Can't pickle local object 'convert_exception_to_response.<locals>.inner'

Discovered via #12442.

[zulipbot]

Hello @zulip/server-testing members, this issue was labeled with the "area: testing-infrastructure" label, so you may want to check it out!