Skip to content

io-it-suite profile: support ability to use other runners #18433

@kennknowles

Description

@kennknowles

Currently, we do not have the ability to run integration tests on clusters of > 1 node using any runners other than dataflow. So we currently do not support the ability to invoke them in io-it-suite.

Once we can run the integration tests on clusters of > 1 node, we'll want to be able to run the IO ITs on them. That involves:

  1. In the io-it-suite profiles, we should have a runner flag or something like it to allow choosing a runner
  2. Add support for them in dpb_service in PerfKit Benchmarker - flesh out dpb_service and beam_integration_benchmark for the newly supported runners.

While we are building this, we should improve the mechanism by which we invoke the direct runner in io-it-suite. Currently, we use forceDirectRunner, which is a maven property that overrides the runner options/profiles to be added to the mvn invocation of io-it. We'll need to talk to the pkb team about how we should better support this since we'll need to make a modification on their side.

cc [~iemejia] [~jasonkuster]

Imported from Jira BEAM-2654. Original Jira may contain additional context.
Reported by: sisk.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions