Skip to content

Conversation

@zentol
Copy link
Contributor

@zentol zentol commented Jan 22, 2018

What is the purpose of the change

This PR modifies the generation of default operator names for window operators to be more concise, and more importantly, stable across job submissions.

Example: SocketTextWordCount

Before:

TriggerWindow(TumblingProcessingTimeWindows(5000), ReducingStateDescriptor{serializer=org.apache.flink.api.java.typeutils.runtime.PojoSerializer@591ae253, reduceFunction=org.apache.flink.streaming.examples.socket.SocketWindowWordCount$1@48974e45}, ProcessingTimeTrigger(), WindowedStream.reduce(WindowedStream.java:241))

After:

Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger, WindowedStream.reduce(WindowedStream.java:243))

Brief change log

  • create shared static utility method to generate names
  • remove state-descriptor from operator name
  • replace TriggerWindow with Window
  • trigger/assigner are now included with their simple class name instead of fully qualified class name + object reference

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

@aljoscha
Copy link
Contributor

Changes look good!

It might make sense to also remove the line number from the string because this might also change in the future. It's not a must, though.

@zentol
Copy link
Contributor Author

zentol commented Jan 22, 2018

How about we replace the line-number with the UDF class name?

@aljoscha
Copy link
Contributor

👍

@zentol
Copy link
Contributor Author

zentol commented Jan 24, 2018

@aljoscha Done, we now include the function name instead. Is a bit more complicated that I had hoped though...

@aljoscha
Copy link
Contributor

Looks good 👍

@zentol
Copy link
Contributor Author

zentol commented Jan 29, 2018

merging.

zentol added a commit to zentol/flink that referenced this pull request Jan 29, 2018
zentol added a commit to zentol/flink that referenced this pull request Jan 29, 2018
zentol added a commit to zentol/flink that referenced this pull request Jan 30, 2018
zentol added a commit to zentol/flink that referenced this pull request Jan 30, 2018
@asfgit asfgit closed this in 7d4bd4b Jan 30, 2018
@zentol zentol deleted the 6464 branch January 30, 2018 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants