Skip to content

Conversation

@markflyhigh
Copy link
Contributor

@markflyhigh markflyhigh commented Apr 16, 2019

Support Python3 performance benchmarks - part 2

Keep use of Perfkit performance infrastructure for Python3 benchmarking due to exist integration with Python pipeline and automated metrics publishing. Part 1 changes are on PerfkitBenchmarker in GoogleCloudPlatform/PerfKitBenchmarker@d0ad984

Changes include in this PR:

  • Move python performance test parameters to new class PythonPerformanceTestConfiguration to make task construction clear and easier to maintain.
  • Create enablePythonPerformanceTest in BeamModulePlugin so that people can enable python performance test in different projects by simply calling this function.
  • Enable performance tests in Python3 project sdks/python/test-suites/dataflow/py3
  • Fix a bug in run_integration_test.sh due to wrong check order.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Python Build Status
Build Status
--- Build Status
Build Status
Build Status --- --- ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@markflyhigh
Copy link
Contributor Author

markflyhigh commented Apr 17, 2019

+R: @udim,
cc: @tvalentyn

@markflyhigh markflyhigh self-assigned this Apr 17, 2019
@udim
Copy link
Member

udim commented Apr 17, 2019

run python postcommit

@markflyhigh markflyhigh removed their assignment Apr 18, 2019
Copy link
Member

@udim udim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but please wait for postcommits to be green before merging

@markflyhigh
Copy link
Contributor Author

markflyhigh commented Apr 19, 2019

Thank you. postcommit failed consistently for a while and got fixed recently (today?). I'll rebase and rerun them.

Would like to merge this PR after Perfkit change merged in master to make the transition status as short as possible.

@markflyhigh
Copy link
Contributor Author

run python postcommit

@markflyhigh
Copy link
Contributor Author

Perfkit corresponding changes are merged in GoogleCloudPlatform/PerfKitBenchmarker@d0ad984. We now can merge this PR after tests pass.

@markflyhigh
Copy link
Contributor Author

Run Java PreCommit

1 similar comment
@markflyhigh
Copy link
Contributor Author

Run Java PreCommit

@markflyhigh
Copy link
Contributor Author

Rebased this PR against master.

@markflyhigh
Copy link
Contributor Author

Run Python_PVR_Flink PreCommit

@markflyhigh markflyhigh merged commit c97e391 into apache:master Apr 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants