[FLINK-4861] [build] Package optional project artifacts

[greghogan]

Package the Flink connectors, metrics, and libraries into subdirectories of a new opt directory in the release/snapshot tarballs.

The following artifacts are packaged by this build:

$ ls build-target/opt/
flink-cep_2.10-1.2-SNAPSHOT.jar
flink-cep-scala_2.10-1.2-SNAPSHOT.jar
flink-gelly_2.10-1.2-SNAPSHOT.jar
flink-gelly-examples_2.10-1.2-SNAPSHOT.jar
flink-gelly-scala_2.10-1.2-SNAPSHOT.jar
flink-metrics-dropwizard-1.2-SNAPSHOT.jar
flink-metrics-ganglia-1.2-SNAPSHOT.jar
flink-metrics-graphite-1.2-SNAPSHOT.jar
flink-metrics-statsd-1.2-SNAPSHOT.jar
flink-ml_2.10-1.2-SNAPSHOT.jar

[greghogan]

@StephanEwen or @rmetzger would you be able to review before today's code freeze?

[StephanEwen]

I can try to give it a shot later...

[StephanEwen]

@greghogan Seems I have a few more days to look at that. Will try to get back at this soon...

[StephanEwen]

Checking this out now...

[StephanEwen]

Okay, the approach of this pull request looks good, but there are two things we need to adjust:

• The fat jar from the flink-ml project contains all dependencies. I think that is because the Maven Assembly plugin does not respect the "provided" status of the core dependencies. We should use the Maven Shade Plugin instead.

• For the same reason, the reporter jars are quite messy. We should fix those as well.

[StephanEwen]

I'll try to address that, then merge this pull request...

[greghogan]

Thanks @StephanEwen. FLINK-4782 may be related in the handling of provided dependencies (external, in that case).

[StephanEwen]

Pull request #3029 is the prerequisite for this pull request to work nicely. Once that is merged, we can merge this one here as is.

[StephanEwen]

Actually, just saw that flink-gelly-scala does not build a "jar-with-dependencies".

Also, gelly and gelly-scala still have the ASM dependency (which seems not necessary any more). We should remove those dependencies, otherwise the "jar with dependency" will have not properly relocated asm classes (and we would need to define more shade profiles)

[StephanEwen]

I have an update with the mentioned changes. Will push that once #3029 gets approval.