Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-8003] Remove Perfkit leftovers #9450

Merged
merged 3 commits into from Sep 20, 2019
Merged

Conversation

@lgajowy
Copy link
Contributor

lgajowy commented Aug 29, 2019

Removes all mentions of PKB in documentation that are out of date. Removes Gradle task that is obsolete since we resigned from using Perfkit. Removes Perfkit word from file-based tests description (makes no sense anymore).

@markflyhigh could you take a look?


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status --- --- Build Status
XLang --- --- --- Build Status --- --- ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@lgajowy lgajowy requested a review from markflyhigh Aug 29, 2019
@lgajowy lgajowy changed the title [BEAM-8003] Remove Perfkit from webside documentation [BEAM-8003] Remove Perfkit leftovers Aug 29, 2019
@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Aug 29, 2019

Run seed job

@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Aug 29, 2019

Run Java TextIO Performance Test HDFS

@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Sep 16, 2019

@markflyhigh pinging kindly. :) Let me know if you have no time to review and I will reassign.

@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Sep 17, 2019

Reassigning: @mwalenia could you take a look?

@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Sep 17, 2019

@aromanenko-dev if you have time could you take a look too?

@aromanenko-dev

This comment has been minimized.

Copy link
Contributor

aromanenko-dev commented Sep 18, 2019

Run Website_Stage_GCS PreCommit

Copy link
Contributor

markflyhigh left a comment

LGTM in general.

##### integrationTest Task {#integration-test-task}
Since `performanceTest` task involved running PerfkitBenchmarker, we can't use it to run the tests manually. For such purposes a more "low-level" task called `integrationTest` was introduced.
#### Running the test {#running-the-test}

This comment has been minimized.

Copy link
@markflyhigh

markflyhigh Sep 18, 2019

Contributor

The section here and below Running Integration Tests on Pull Requests {#running-on-pull-requests} seems two ways of running the tests. Should they be in same level? Also this seems focus on running tests with local Gradle, but not very clear to me.

This comment has been minimized.

Copy link
@lgajowy

lgajowy Sep 19, 2019

Author Contributor

Thanks @markflyhigh! I don't think they should. This paragraph describes the way to run a particular test alone. It's a part of a more highlevel paragraph about running any test on your machine. The other describes the ability of phrase triggering using Beam's CI in general.

I added some sentences to make this more clear. Let me know if this was helpful. :)

Copy link
Contributor

aromanenko-dev left a comment

LGTM, just a minor note

integrationTest is a dedicated gradle task for running IO integration tests.

This comment has been minimized.

Copy link
@aromanenko-dev

aromanenko-dev Sep 19, 2019

Contributor

nit: I suggest to select integrationTest with grave accents `integrationTest`, so it will like this integrationTest

This comment has been minimized.

Copy link
@lgajowy

lgajowy Sep 19, 2019

Author Contributor

👍

@lgajowy lgajowy force-pushed the lgajowy:BEAM-8003 branch from 2dbb738 to 23605d8 Sep 19, 2019
@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Sep 19, 2019

@markflyhigh @aromanenko-dev could you take another look?

@tvalentyn

This comment has been minimized.

Copy link
Contributor

tvalentyn commented Sep 19, 2019

We also have Perfkit leftovers in https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_PerformanceTests_Python.groovy - do we have plan to move those benchmarks off of PKB?

@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Sep 20, 2019

@tvalentyn yes, thank you for asking. Python tests are the last ones that use PKB.

There is a common ticket for removing Perfkit on Jira: BEAM-7772
Removing PKB from Python suites is tracked here (subtask): BEAM-7774

@aromanenko-dev

This comment has been minimized.

Copy link
Contributor

aromanenko-dev commented Sep 20, 2019

Run Portable_Python PreCommit

@aromanenko-dev

This comment has been minimized.

Copy link
Contributor

aromanenko-dev commented Sep 20, 2019

LGTM

@aromanenko-dev aromanenko-dev merged commit bb6f9ed into apache:master Sep 20, 2019
13 checks passed
13 checks passed
CommunityMetrics ("Run CommunityMetrics PreCommit") SUCCESS
Details
Go ("Run Go PreCommit") SUCCESS
Details
Java ("Run Java PreCommit") SUCCESS
Details
JavaBeamZetaSQL ("Run JavaBeamZetaSQL PreCommit") SUCCESS
Details
JavaPortabilityApi ("Run JavaPortabilityApi PreCommit") SUCCESS
Details
Java_Examples_Dataflow ("Run Java_Examples_Dataflow PreCommit") SUCCESS
Details
Portable_Python ("Run Portable_Python PreCommit") SUCCESS
Details
Python ("Run Python PreCommit") SUCCESS
Details
Python_PVR_Flink ("Run Python_PVR_Flink PreCommit") SUCCESS
Details
RAT ("Run RAT PreCommit") SUCCESS
Details
Spotless ("Run Spotless PreCommit") SUCCESS
Details
Website ("Run Website PreCommit") SUCCESS
Details
Website_Stage_GCS ("Run Website_Stage_GCS PreCommit") SUCCESS
Details
@lgajowy

This comment has been minimized.

Copy link
Contributor Author

lgajowy commented Sep 20, 2019

Thank you all! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.