Skip to content

[BEAM-8139] Implement Spark portable job jars.#10287

Merged
ibzib merged 3 commits intoapache:masterfrom
ibzib:spark-jar
Dec 12, 2019
Merged

[BEAM-8139] Implement Spark portable job jars.#10287
ibzib merged 3 commits intoapache:masterfrom
ibzib:spark-jar

Conversation

@ibzib
Copy link

@ibzib ibzib commented Dec 4, 2019

This is a pretty straightforward copy/paste/find/replace of the Flink implementation.

For now, jars must be created by Java. The next step would be to create an equivalent for FlinkUberJarJobServer, though we probably want to fix that first before porting it to Spark.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status
Build Status
--- --- Build Status
XLang --- --- --- Build Status --- --- ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status
Build Status
Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@ibzib ibzib requested review from mxm and robertwb December 4, 2019 19:30
@ibzib
Copy link
Author

ibzib commented Dec 4, 2019

Test failures (:runners:direct-java:needsRunnerTests and :runners:samza:test) look unrelated.

Copy link
Contributor

@mxm mxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Perhaps let @iemejia also have a look.

@ibzib ibzib merged commit 8cf22ec into apache:master Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants