Skip to content

[BEAM-7772] remove pkb from JDBCIOIT, HadoopFormatIOIT and hdfs tests #9315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
lgajowy merged 7 commits into from
Aug 19, 2019
Merged

[BEAM-7772] remove pkb from JDBCIOIT, HadoopFormatIOIT and hdfs tests #9315

lgajowy merged 7 commits into from
Aug 19, 2019

Conversation

@lgajowy
Copy link
Contributor

@lgajowy lgajowy commented Aug 12, 2019 •
edited
Loading

Removes perfkit usage from JDBC, HadoopFromat and HDFS file based ioit tests.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status		 Build Status Build Status Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status		 --- Build Status
Build Status		 Build Status --- --- Build Status
XLang --- --- --- Build Status --- --- ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 12, 2019

Run seed job

1 similar comment
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java JdbcIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java HadoopFormatIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java JdbcIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java TextIO Performance Test HDFS

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from 5f747d7 to 88e7a3c Compare August 13, 2019 11:28
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java TextIO Performance Test HDFS

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java TextIO Performance Test HDFS

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from 13e371d to 1d4d028 Compare August 13, 2019 14:34
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run Java TextIO Performance Test HDFS

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from 1d4d028 to 6e53bb1 Compare August 13, 2019 15:10
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy lgajowy changed the title [BEAM-7772] remove pkb from JDBCIOIT, HadoopFormatIOIT [BEAM-7772] remove pkb from JDBCIOIT, HadoopFormatIOIT and hdfs tests Aug 13, 2019
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

Run seed job

@lgajowy lgajowy marked this pull request as ready for review August 13, 2019 15:56
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 13, 2019

@kkucharc @mwalenia could you take a look?

kkucharc
kkucharc reviewed Aug 14, 2019
View reviewed changes
.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
runner : 'DataflowRunner',
project : 'apache-beam-testing',
tempRoot : 'gs://temp-storage-for-perf-tests',
hdfsConfiguration: /[{\\\"fs.defaultFS\\\":\\\"hdfs:$${hostName}:9000\\\",\\\"dfs.replication\\\":1}]/,
Copy link
Contributor

@kkucharc kkucharc Aug 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am impressed by this escaping 😮

kkucharc
kkucharc approved these changes Aug 14, 2019
View reviewed changes
Copy link
Contributor

@kkucharc kkucharc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks ok for me 👍I saw that you widely tested it, that's great. Thank you for meaningful commit messages and descriptions! 👏

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run seed job

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from 2601a8a to a634dfd Compare August 19, 2019 10:02
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java TextIO Performance Test HDFS

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java TextIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java JdbcIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java HadoopFormatIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java ParquetIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run seed job

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from a634dfd to ee5df43 Compare August 19, 2019 10:21
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java HadoopFormatIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java JdbcIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java ParquetIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java TextIO Performance Test HDFS

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java JdbcIO Performance Test

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from ee5df43 to 5001ae9 Compare August 19, 2019 11:31
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run seed job

@lgajowy lgajowy force-pushed the BEAM-7772-remove-pkb branch from 5001ae9 to e9cf1f4 Compare August 19, 2019 11:44
@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run seed job

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java JdbcIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java HadoopFormatIO Performance Test

@lgajowy
Copy link
Contributor Author

lgajowy commented Aug 19, 2019

Run Java TextIO Performance Test HDFS

@mwalenia
Copy link
Member

mwalenia commented Aug 19, 2019 •
edited
Loading

@lgajowy LGTM, thank you!

@lgajowy lgajowy merged commit c8eff45 into apache:master Aug 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kkucharc kkucharc kkucharc approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lgajowy @mwalenia @kkucharc