Skip to content

The Java SDK Harness and runners-core-construction payloads should be consistent #18702

@kennknowles

Description

@kennknowles

Right now the Java SDK harness expects DoFnInfo instances within the payload of all DoFns it is provided, while the pipeline translation/ParDoTranslation/etc assumes that the ParDoTranslation channel will be used to extract useful values from the serialized form (potentially by converting to a ParDoLike). As such, a Java ParDo serialized with the Java ParDoTranslation class cannot be provided in that form to the Java SDK harness. Instead, that node has to be rewritten to include a DoFnInfo in the payload.

This seems unreasonable, and the two should be brought in line to use the same serialized representation.

Imported from Jira BEAM-3365. Original Jira may contain additional context.
Reported by: tgroh.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions