DataflowRunner does not have enough unit test coverage. This resulted in a silent failure where dataflow job graph was malformed for UI to display but job still completed (https://pantheon.corp.google.com/dataflow/job/2017-03-31_21_56_13-6233023862008864856?project=apache-beam-testing&organizationId=433637338589)
It was a simple fix (#2429) that could be caught by a unit test to check step names are not empty.
Also note that it is possible to use dataflow_job_file flag to create job files only without actually running jobs for the purpose of unit testing.
Imported from Jira BEAM-1884. Original Jira may contain additional context.
Reported by: altay.
DataflowRunner does not have enough unit test coverage. This resulted in a silent failure where dataflow job graph was malformed for UI to display but job still completed (https://pantheon.corp.google.com/dataflow/job/2017-03-31_21_56_13-6233023862008864856?project=apache-beam-testing&organizationId=433637338589)
It was a simple fix (#2429) that could be caught by a unit test to check step names are not empty.
Also note that it is possible to use
dataflow_job_fileflag to create job files only without actually running jobs for the purpose of unit testing.Imported from Jira BEAM-1884. Original Jira may contain additional context.
Reported by: altay.