Skip to content

[BEAM-3573] Move GCPIO test utilities to main jar#6838

Merged
kennknowles merged 2 commits intoapache:masterfrom
kennknowles:bqtornadoes-test-deps
Oct 29, 2018
Merged

[BEAM-3573] Move GCPIO test utilities to main jar#6838
kennknowles merged 2 commits intoapache:masterfrom
kennknowles:bqtornadoes-test-deps

Conversation

@kennknowles
Copy link
Member

These test utilities are used by clients to test their code. They are not isolated to the test suite jar.


Follow this checklist to help us incorporate your contribution quickly and easily:

  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status Build Status Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

@kennknowles
Copy link
Member Author

There is some hope that eliminating bad test jar deps might mitigate https://issues.apache.org/jira/browse/BEAM-5035, https://issues.apache.org/jira/browse/BEAM-5207, https://issues.apache.org/jira/browse/BEAM-5116. In any case it is the right direction so no harm in moving to find out how much it helps.

@kennknowles kennknowles force-pushed the bqtornadoes-test-deps branch from 380785d to 3921f49 Compare October 26, 2018 03:16
@kennknowles
Copy link
Member Author

R: @apilloud

I believe you did the deepest digging on this parallel build issue. From the issues and gists, I'm not 100% certain it all comes back to this, but I'm not 100% certain it doesn't. Our other modules have test deps that can be just "test runtime" deps. But I'm pretty sure this is the only hard dependency on a class in another test jar for any transitive dep of the tasks I saw failing.

@apilloud
Copy link
Member

I did little to no digging beyond disabling parallel tests to fix flakes. This looks like it might fix the root cause. LGTM.

@kennknowles
Copy link
Member Author

run java postcommit

@kennknowles
Copy link
Member Author

The failure in postcommit is the same from #6864. We have tests in the Dataflow worker codebase that are not thread safe. I will drop that commit, then this should be g2g.

@kennknowles kennknowles force-pushed the bqtornadoes-test-deps branch from 32897e8 to a8546bc Compare October 29, 2018 17:11
@kennknowles
Copy link
Member Author

run java postcommit

@kennknowles kennknowles merged commit 80a141a into apache:master Oct 29, 2018
@kennknowles kennknowles deleted the bqtornadoes-test-deps branch November 9, 2018 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants