Skip to content

Python SDK does not correctly set the serialized_fn of WindowInto when translating to the Dataflow pipeline description #19034

@kennknowles

Description

@kennknowles

serialzed_fn at the bottom should be reference to the pipeline proto ptransform id representing this function.


    {
      "kind": "ParallelDo", 
      "name": "s4", 
      "properties": {
        "display_data":
[
          {
            "key": "fn", 
            "label": "Transform Function", 
           
"namespace": "apache_beam.transforms.core.WindowInto", 
            "shortValue": "WindowIntoFn", 

           "type": "STRING", 
            "value": "apache_beam.transforms.core.WindowIntoFn"
   
      }
        ], 
        "non_parallel_inputs": {}, 
        "output_info": [
          {
 
          "encoding": {
              "@type": "kind:windowed_value", 
              "component_encodings":
[
                {
                  "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",

                  "component_encodings": [
                    {
                      "@type":
"FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",

                      "component_encodings": []
                    }, 
                    {

                     "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/",

                      "component_encodings": []
                    }
                  ], 
  
               "is_pair_like": true
                }, 
                {
                  "@type":
"kind:global_window"
                }
              ], 
              "is_wrapper": true
     
      }, 
            "output_name": "out", 
            "user_name": "assert_that/WindowInto(WindowIntoFn).out"

         }
        ], 
        "parallel_input": {
          "@type": "OutputReference", 
     
    "output_name": "out", 
          "step_name": "s2"
        }, 
        "serialized_fn": "<string
of 328 bytes>", 
        "user_name": "assert_that/WindowInto(WindowIntoFn)"
      }
    }, 

Imported from Jira BEAM-4581. Original Jira may contain additional context.
Reported by: lcwik.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions