PortableRunnerTestWithExternalEnv.test_pardo_timers flaky

[damccorm]

In python precommit.

Looks like there's also a bug in error handling. IIUC, this call is intended to be a "cast" but really it's calling an arbitrary constructor, which may take different arguments than a
BaseException. 

beam/sdks/python/apache_beam/runners/worker/sdk_worker.py

Line 1105 in 4559c75

raise t(v).with_traceback(tb)

cc[~tvalentyn] [~yoshiki.obata]
 
https://ci-beam.apache.org/job/beam_PreCommit_Python_Phrase/2755/testReport/junit/apache_beam.runners.portability.portable_runner_test/PortableRunnerTestWithExternalEnv/test_pardo_timers/
 
E RuntimeError: Pipeline job-721c3984-7f9c-4991-8473-c7d8c02f4d13 failed in state FAILED: Error running pipeline. E Traceback (most recent call last): E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/local_job_service.py", line 274, in _run_job E self._pipeline_proto) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 206, in run_via_runner_api E return self.run_stages(stage_context, stages) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 385, in run_stages E runner_execution_context, bundle_context_manager) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 653, in _run_stage E bundle_manager)) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 770, in _run_bundle E data_input, data_output, input_timers, expected_timer_output) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 1120, in process_bundle E raise RuntimeError(result.error) E RuntimeError: Traceback (most recent call last): E File "apache_beam/runners/common.py", line 1258, in apache_beam.runners.common.DoFnRunner.process_user_timer E self.do_fn_invoker.invoke_user_timer( E File "apache_beam/runners/common.py", line 528, in apache_beam.runners.common.DoFnInvoker.invoke_user_timer E self.output_processor.process_outputs( E File "apache_beam/runners/common.py", line 1368, in apache_beam.runners.common._OutputProcessor.process_outputs E for result in results: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner_test.py", line 366, in process_timer E if state.read() == 0: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 481, in read E return self._combinefn.extract_output(self._read_accumulator()) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 473, in _read_accumulator E self._underlying_bag_state.read()) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/transforms/core.py", line 1038, in merge_accumulators E return [self._fn(_ReiterableChain(accumulators), *args, **kwargs)] E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/transforms/core.py", line 976, in iter E for iterable in self.iterables: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 521, in iter E for elem in self.first: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 353, in iter E self._state_handler.blocking_get(self._state_key, self._coder_impl)) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1194, in blocking_get E self._partially_cached_iterable(state_key, coder)) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1300, in _partially_cached_iterable E data, continuation_token = self._underlying.get_raw(state_key, None) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1067, in get_raw E continuation_token=continuation_token))) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1105, in _blocking_request E raise t(v).with_traceback(tb) E TypeError: init() missing 3 required positional arguments: 'call', 'response_deserializer', and 'deadline'

Imported from Jira BEAM-12794. Original Jira may contain additional context.
Reported by: ibzib.

[damccorm]

This hasn't caused CI to fail within the current history (with the exception of this one where it looks like Jenkins itself may have flaked out and a whole suite all failed), and running it 100 times locally it didn't fail once (for i in {1..100} ; do (pytest -v apache_beam/runners/portability/fn_api_runner/fn_runner_test.py::FnApiRunnerTest::test_pardo_timers) ; done )