[BEAM-8941] Implement simple DSL for load tests

[pawelpasterz]

Implement simple DSL for load tests. Probably first iteration of this PR.

Changes:

• config -- creates configuration map from given closure (example below)
• templateConfig -- create reusable LoadTestConfig instance (no need to create whole map for every single test), specific settings can be changed for every configuration map made from template (title, input options etc)
• fromTemplate -- creates configuration map from given template, specific settings can be changed
• validation for every returned map is performed (ex Flink runner requires environmentConfig to be set, test can not be launched without runner etc)

def loadTestConfigurations = { datasetName ->
    final def template = templateConfig {
        test 'apache_beam.testing.load_tests.group_by_key_test:GroupByKeyTest.testGroupByKey'
        dataflow()
        pipelineOptions {
            python()
            project 'apache-beam-testing'
            tempLocation 'gs://temp-storage-for-perf-tests/loadtests'
            publishToBigQuery true
            metricsDataset datasetName
            numWorkers 5
            autoscalingAlgorithm "NONE"
            specificParameters([
                fanout: 1,
                iterations: 1
            ]}
        }
    }
    [
            fromTemplate(template) {
                title 'GroupByKey Python Load test 2GB of 10B records'
                pipelineOptions {
                    jobName 'load-tests-python-dataflow-batch-gbk-1-'
                    metricsTable 'python_dataflow_batch_gbk_1'
                    inputOptions {
                        numRecords 200000000
                        keySize 1
                        valueSize 9
                    }

                }
            },
            fromTemplate(template) {
                title 'GroupByKey Python Load test 2GB of 100kB records'
                pipelineOptions {
                    jobName 'load-tests-python-dataflow-batch-gbk-3-'
                    metricsTable 'python_dataflow_batch_gbk_3'
                    inputOptions {
                        numRecords 2000
                        keySize 100000
                        valueSize 900000
                    }

                }
            },
            fromTemplate(template) {
                title 'GroupByKey Python Load test fanout 4 times with 2GB 10-byte records total'
                pipelineOptions {
                    jobName 'load-tests-python-dataflow-batch-gbk-4-'
                    metricsTable 'python_dataflow_batch_gbk_4'
                    inputOptions {
                        numRecords 5000000
                        keySize 10
                        valueSize 90
                    }
                    specificParameters([
                        fanout: 5
                    ]}
                }
            }
    ]
}
//compatible with current, map based implementation
loadTestsBuilder.loadTests(delegate, CommonTestProperties.SDK.PYTHON, loadTestConfigurations(datasetName), "GBK", "batch")

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---		---	---
Java
Python		---			---	---
XLang	---	---	---		---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[pawelpasterz]

@kamilwu @mwalenia
Let me know what do you think. Thanks!

[kamilwu]

Thanks @pawelpasterz for your contribution! I've left some comments. In general, I believe this is a huge step forward in making testing infrastructure more friendly and less error-prone. It's good that we can abort seed job in case of some simple errors: type mismatch and lack of a required parameter.

[kamilwu]

Please avoid squashing unreviewed changes and force-pushing the branch. Instead, create an additional "fixup" commit: https://beam.apache.org/contribute/#make-the-reviewers-job-easier

[pawelpasterz]

@kamilwu I've implemented changes you requested. I think all possible parameters for load tests are included now. Let me know what do you think.
And thank for for pointing me redundant squashing.

[kamilwu]

Thanks, LGTM. Let's ask @kkucharc if she can help with the merge.

[kkucharc]

Thank you @pawelpasterz for this refactor :)
I left my comments.
Besides of that I've got two additional remarks:

• it would be great to see how SMOKE test would change with this refactor
• I am missing adding '_PR' suffix in tests which are phase triggered (please compare with this ). Should it be here?

[kkucharc]

Do you have any load tests in Spark or Flink (non-portable)? I am asking because those two options can be added later. My main concern is here if we are sure if someone will actually run test with Spark/Flink option if it won't fail.

[pawelpasterz]

TBH I looked at CommonTestProperties class and picked Runner values. Since I am not that much familiar with code yet, my implementation (the one above) can be wrong. Should I limit it to dataflow and portable? (at least for the time being)

[pawelpasterz]

Hey @kkucharc I've made some changes and implemented smoke test. Let me know what do you think.
Regarding your question related with _PR suffix -- this dsl provides only configuration map which is then passed to LoadTestBuilder (as it is now). So there is no need to add suffix here.

[kkucharc]

Thanks @pawelpasterz Let's check how it behaves on Jenkins 😄

[kkucharc]

run seed job

[kkucharc]

Run Java Load Tests DSL Smoke

[pawelpasterz]

Hey @kkucharc , I fixed typo, should work now :) Please rerun seed job and smoke test, thanks!

[kkucharc]

run seed job

[kkucharc]

Run Java Load Tests DSL Smoke

[kkucharc]

run seed job

[kkucharc]

Run Java Load Tests DSL Smoke

[kkucharc]

Run Java Load Tests DSL Smoke

[kkucharc]

run seed job

[kkucharc]

Run Java Load Tests DSL Smoke

[kkucharc]

@pawelpasterz looks like everything is fine now 🎉 Can you squash commits?

[pawelpasterz]

@kkucharc it is done