Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

android/xts: add support for vts module level static sharding #422

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

chase-qi
Copy link
Contributor

It can useful when test setup is cheap and many DUTs are available for test runs in parallel.

Signed-off-by: Chase Qi chase.qi@linaro.org

It can useful when test setup is cheap and many DUTs are available for
test runs in parallel.

Signed-off-by: Chase Qi <chase.qi@linaro.org>
@chase-qi
Copy link
Contributor Author

Job example:

Run by a whole ltp module https://validation.linaro.org/scheduler/job/3848967, 1121 test results reported.

Shard ltp module into 3 shards.
shard3-1 https://validation.linaro.org/scheduler/job/3848962, 475 test results reported.
shard3-2 https://validation.linaro.org/results/3848964, 428 test results reported.
shard3-3 https://validation.linaro.org/scheduler/job/3849093 218 test results reported.
The overall test results reported by the 3 shards equal to 1121 which is the same as test run without sharding.

@liuyq @roxell @ivoire would you please help to review this MR?

tree = etree.parse(config)
root = tree.getroot()

# vts test modules like ltp and kselftest modules contain
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

splitting cts test config is a different story. I looked into the big cts test module testcases/CtsLibcoreTestCases/CtsLibcoreTestCases.config, I don't see individual tests in the config.

    <test class="com.android.compatibility.testtype.LibcoreTest" >
        <option name="package" value="android.libcore.cts" />
        <option name="instrumentation-arg" key="filter"
                value="com.android.cts.core.runner.ExpectationBasedFilter" />
        <option name="core-expectation" value="/knownfailures.txt" />
        <option name="virtual-device-core-expectation" value="/virtualdeviceknownfailures.txt" />
        <option name="runtime-hint" value="45m"/>
        <!-- 20x default timeout of 600sec -->
        <option name="shell-timeout" value="12000000"/>
        <option name="hidden-api-checks" value="false"/>
        <option name="device-listeners" value="org.conscrypt.ConscryptInstrumentationListener" />
        <option name="instrumentation-arg" key="conscrypt_sslsocket_implementation" value="engine" />
    </test>

More investigation is required to split cts test module. Not in this MR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant