[FLINK-18802][formats] Package uber jar including all dependencies for flink-avro

[PatrickRen]

What is the purpose of the change

This pull request changes the pom.xml of flink-avro module in order to create a uber jar in packaging phase, including dependencies such as avro, jackson-core-asl, jackson-mapper-asl and joda-time.

Brief change log

• Add another execution in maven-assembly-plugin to create a jar-with-dependencies
• Remove the "provided" scope and "optional" tag of joda-time dependency to include it in the uber jar

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (yes / no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
• The S3 file system connector: (no)

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (not applicable)

[flinkbot]

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit 69e9338 (Thu Aug 06 02:00:19 UTC 2020)

Warnings:

• 1 pom.xml files were touched: Check for build and licensing issues.
• No documentation files were touched! Remember to keep the Flink docs up to date!
• This pull request references an unassigned Jira ticket. According to the code contribution guide, tickets need to be assigned before starting with the implementation work.

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

• ❓ 1. The [description] looks good.
• ❓ 2. There is [consensus] that the contribution should go into to Flink.
• ❓ 3. Needs [attention] from.
• ❓ 4. The change fits into the overall [architecture].
• ❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
• @flinkbot approve all to approve all aspects
• @flinkbot approve-until architecture to approve everything until architecture
• @flinkbot attention @username1 [@username2 ..] to require somebody's attention
• @flinkbot disapprove architecture to remove an approval you gave earlier

[flinkbot]

CI report:

• 766339d Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run travis re-run the last Travis build
• @flinkbot run azure re-run the last Azure build

[becketqin]

LGTM. We probably also want to have uber jars for other formats as well.
@wuchong Do you want to take a look?

[PatrickRen]

Looks like CI failed. Fixed in 766339d

[twalthr]

As I mentioned in the JIRA issue, we should have consensus on adding a default Avro version. I'm fine with adding one for the SQL Client JAR file because we will always use GenericRecord there. However, for DataStream API jobs users often use SpecificRecord which might compile against a specific Avro version.

Also, for SQL Client JAR files with GenericRecord we probably don't need JodaTime anymore. The newest Avro version is JodaTime free, no?
https://issues.apache.org/jira/browse/AVRO-2079

[PatrickRen]

Thanks a lot @twalthr ~ What about upgrading Avro version to 1.9.0 so that we can get rid of joda-time, and packaging it into sql-jar as the default Avro version?

[dawidwys]

Hey @PatrickRen,
I think it would be fine upgrading the avro version for the sql jar. We could even think of upgrading it straight to the 1.10 version. Again I very much agree with @twalthr that we can do that for the sql jar only.

In case of the {{flink-avro}} module the situation is not that easy. Users might've generated their classes against specific version. Moreover they might want to use the version of avro that comes with hadoop they're using. Therefore they should have an easy way to replace the avro version.

[dawidwys]

I am closing this PR because of inactivity and because it is subsumed by #13373 . Feel free to reopen if you think otherwise.