Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-79] Merge master into gearpump-runner #3292

Merged
merged 254 commits into from Jun 12, 2017

Conversation

manuzhang
Copy link
Contributor

@manuzhang manuzhang commented Jun 5, 2017

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

  • Make sure the PR title is formatted like:
    [BEAM-<Jira issue #>] Description of pull request
  • Make sure tests pass via mvn clean verify.
  • Replace <Jira issue #> in the title with the actual Jira issue
    number, if there is one.
  • If this contribution is large, please file an Apache
    Individual Contributor License Agreement.

dhalperi and others added 30 commits May 18, 2017 13:07
Add ParDoPayloadTranslator to PTransformTranslator
We use the same test table name in two tests: one that expects the table not to exist, another that
creates the table. Obviously, the correctness of this will depend on the order in which unit tests
are executed.

Fix the flake by using different and more explicit test names.
…ugin to inherit build order from root pom.xml
…ugin to inherit build order from root pom.xml

This closes apache#3152
 - [x] Make sure the PR title is formatted like:
   `[BEAM-<Jira issue #>] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
       Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
       number, if there is one.
 - [x] If this contribution is large, please file an Apache
       [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).

---
…n root pom.xml within maven archetype poms

This closes apache#3159
@manuzhang manuzhang force-pushed the sync-master branch 2 times, most recently from 318b967 to bdade1f Compare June 5, 2017 04:40
@coveralls
Copy link

Coverage Status

Changes Unknown when pulling c9aac96 on manuzhang:sync-master into ** on apache:gearpump-runner**.

@aaltay
Copy link
Member

aaltay commented Jun 5, 2017

R: @kennknowles

@manuzhang
Copy link
Contributor Author

@kennknowles Despite failing tests, this PR tries to catch up with master, activates pre-commit tests and includes several bug fixes. Can we move forward and fix the tests in following PRs ?

@kennknowles
Copy link
Member

Yes, that is a good point. Let's get this in since there have been so many changes on master. Since the tests were probably already troubled by being deactivated it isn't like we are introducing new problems.

@asfgit asfgit merged commit 7653e7e into apache:gearpump-runner Jun 12, 2017
asfgit pushed a commit that referenced this pull request Jun 12, 2017
Adjustments to gearpump-runner:

  Fix side input handling in DoFnFunction
  Respect WindowFn#getOutputTime in gearpump-runner
  Activate Gearpump local-validates-runner-tests in precommit
  Update against master changes

Merged from master:

  [BEAM-972] Add more unit test to Gearpump runner
  [BEAM-972] Add unit tests to Gearpump runner
  [BEAM-79] Fix gearpump-runner merge conflicts and test failure
  enable ParDoTest
  [BEAM-79] Add SideInput support for GearpumpRunner
  [BEAM-79] Support merging windows in GearpumpRunner
  [BEAM-79] Fix PostCommit test confs for Gearpump runner
  note thread is interrupted on InterruptedException
  Remove cache for Gearpump on travis
  reduce timeout to wait for result
  fix ParDo.BoundMulti translation
  return encoded key for GroupByKey translation
  support OutputTimeFn
  update to latest gearpump dsl function interface
  fix group by window
  activate ROS on Gearpump by default
  update ROS configurations
  [BEAM-1180] Implement GearpumpPipelineResult
  [BEAM-79] Upgrade to beam-0.5.0-incubating-SNAPSHOT
  [BEAM-79] Update to latest Gearpump API
  Fix NoOpAggregatorFactory
  Remove print to stdout
  Skip window assignment when windows don't change
  Add Window.Bound translator
  Upgrade Gearpump version
  [BEAM-79] fix gearpump runner build failure
  [BEAM-79] update GearpumpPipelineResult
  [BEAM-79] Port Gearpump runner from OldDoFn to new DoFn
  upgrade gearpump-runner to 0.4.0-incubating-SNAPSHOT
  remove "pipeline" in runner name
  post-merge fix
  [BEAM-79] fix integration-test failure
  fix import order
  !fixup Minor javadoc clean-up
  Added even more javadoc to TextIO#withHeader and TextIO#withFooter (2).
  Added even more javadoc to TextIO#withHeader and TextIO#withFooter.
  Added javadoc to TextIO#withHeader and TextIO#withFooter.
  Reverted header and footer to be of type String.
  Revised according to comments following a code review.
  Add header/footer support to TextIO.Write
  [BEAM-242] Enable and fix checkstyle in Flink runner examples
  Remove timeout in JAXBCoderTest
  Be more accepting in UnboundedReadDeduplicatorTest
  BigQuery: limit max job polling time to 1 minute
  [BEAM-242] Enable checkstyle and fix checkstyle errors in Flink runner
  [BEAM-456] Add MongoDbIO
  FluentBackoff: a replacement for a variety of custom backoff implementations
  Remove the DataflowRunner instructions from examples
  Put classes in runners-core package into runners.core namespace
  Delegate populateDipslayData to wrapped combineFn's
  Fixed Combine display data
  Cloud Datastore naming clean-up
  DatastoreIO SplitQueryFn integration test
  Add Latest CombineFn and PTransforms
  Remove empty unused method in TestStreamEvaluatorFactory
  Test that multiple instances of TestStream are supported
  Correct some accidental renames
  Fix condition in FlinkStreamingPipelineTranslator
  Address comments of Flink Side-Input PR
  [BEAM-569] Define maxNumRecords default value to Long.MAX_VALUE in JmsIO
  Add LeaderBoardTest
  take advantage of setup/teardown for KafkaWriter
  Returned KafkaIO getWatermark log line in debug mode
  [BEAM-572] Remove Spark Reference in WordCount
  Update Dataflow Container Version
  [BEAM-313] Provide a context for SparkRunner
  DataflowRunner: get PBegin from PInput
  [BEAM-592] Fix SparkRunner Dependency Problem in WordCount
  Fix javadoc in Kinesis
  Organize imports in Kinesis
  kinesis: a connector for Amazon Kinesis
  [BEAM-589] Fixing IO.Read transformation
  Query latest timestamp
  travis.yml: disable updating snapshots
  Added support for reporting aggregator values to Spark sinks
  [BEAM-294] Rename dataflow references to beam
  Modified BigtableIO to use DoFn setup/tearDown methods instead of startBundle/finishBundle
  checkstyle: prohibit API client repackaged Guava
  Make WriteTest more resilient to Randomness
  Update DoFn javadocs to remove references to OldDoFn and Dataflow
  [BEAM-545] Promote JobName to PipelineOptions
  Move the samples data to gs://apache-beam-samples/
  Cleanup some javadoc that referring Dataflow
  BigQueryIO.Write: raise size limit to 11 TiB
  Optimize imports
  Update checkstyle.xml to put all imports in one group
  Fix Exception Unwrapping in TestFlinkRunner
  Make ParDoLifecycleTest Serializable to Fix Test with TupleTag
  Use AllPanes as the PaneExtractor in IterableAssert
  Fix combine tests with Accumulation Mode
  Enable Flink Streaming Runner RunnableOnService tests
  Fix Flink Runner Pom for Batch RunnableOnService tests
  Don't Suppress Throwable in PAssert in Streaming Mode
  ...
@manuzhang manuzhang deleted the sync-master branch June 13, 2017 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet