[BEAM-5917] Update Flink to 1.5.5

[mxm]

The force-shading module issue is strange but it doesn't seem to be published: https://repo.maven.apache.org/maven2/org/apache/flink/force-shading/

I've reported this on the Flink mailing list.

CC @angoenka @tweise

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---	---	---	---	---
Java
Python		---			---	---	---

[tweise]

Let's put this on hold until the Flink issue is sorted out. I prefer generally that we wait until things are fully baked and we have done some more verification, the Flink release has been only out for a day..

[mxm]

IMHO updating to the latest bugfix release will give us only benefits. For minor releases I agree that it makes sense to wait a bit until it stabilizes.

The Flink issue is reported here: https://issues.apache.org/jira/browse/FLINK-10732

[tweise]

We can safely defer this change until after the Flink release is fixed, at which time it will be just a version number change with no other workarounds.

It would also be nice to run something on a Flink 1.5.5 cluster, if you had not already done that.

That can be done without any change in Beam (we are using 1.5.4 at Lyft currently).

[mxm]

The problem with the force-shading dependency has been resolved.

It would also be nice to run something on a Flink 1.5.5 cluster, if you had not already done that.

Typically, all hotfix versions of Flink are compatible with each other (in the absence of bugs). There are integration tests which run as part of this PR. In fact, I've already been using Flink 1.5.4 and 1.5.5 to run the Beam code based on Flink 1.5.3. I haven't experienced any problems. But let's run ValidatesRuner and PortableValidatesRunner.

[mxm]

Run Flink ValidatesRunner

[mxm]

Run Python Flink ValidatesRunner

[mxm]

Run Java Flink PortableValidatesRunner

[mxm]

All tests are passing. Also ran a manual cluster test.