Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Update automated/README using markdown syntax - Move the file out as repo README Signed-off-by: Chase Qi <chase.qi@linaro.org>
- Loading branch information
Showing
2 changed files
with
109 additions
and
74 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
# Test Definitions | ||
A test suite works with and without [LAVA](http://lavasoftware.org/). The following two | ||
sets of automated tests are supported. | ||
|
||
- `automated/linux/` | ||
- `automated/android/` | ||
|
||
For each test case, both the test script and the corresponding test definition files are | ||
provided in the same folder and are named after the test case name. Test scripts are | ||
self-contained and work independently. Test definition files in YAML format are provided | ||
for test runs with local test-runner and within LAVA. | ||
|
||
## Installation | ||
Installing the latest development version: | ||
|
||
git clone https://github.com/Linaro/test-definitions | ||
cd ./test-definitions | ||
. ./automated/bin/setenv.sh | ||
pip install -r ${REPO_PATH}/automated/utils/requirements.txt | ||
|
||
If the above succeeds, try: | ||
|
||
test-runner -h | ||
|
||
## Running test | ||
### Running test script | ||
#### linux | ||
|
||
cd ./automated/linux/smoke/ | ||
./smoke.sh | ||
|
||
Skip package installation: | ||
|
||
./smoke.sh -s true | ||
|
||
#### android | ||
|
||
cd ./automated/android/dd-wr-speed/ | ||
./dd-wr-speed.sh | ||
|
||
Specify SN when more than one device connected: | ||
|
||
./dd-wr-speed.sh -s "serial_no" | ||
|
||
Specify other params: | ||
|
||
./dd-wr-speed.sh -i "10" -p "/dev/block/mmcblk1p1" | ||
|
||
### Using test-runner | ||
#### single test run | ||
|
||
test-runner -d ./automated/linux/smoke/smoke.yaml | ||
|
||
skip package install: | ||
|
||
test-runner -d ./automated/linux/smoke/smoke.yaml -s | ||
|
||
#### running test plan | ||
|
||
Run a set of tests defined in agenda file: | ||
|
||
test-runner -p ./plans/linux-example.yaml | ||
|
||
Apply test plan overlay to skip, amend or add tests: | ||
|
||
test-runner -p ./plans/linux-example.yaml -O test-plan-overlay-example.yaml | ||
|
||
## Collecting result | ||
|
||
### Using test script | ||
Test script normally puts test log and parsed results to its own `output` directory. e.g. | ||
|
||
automated/linux/smoke/output | ||
|
||
### Using test-runner | ||
test-runner needs a separate directory outside the repo to store test and result files. | ||
The directory defaults to `$HOME/output` and can be changed with `-o <dir>`. test-runner | ||
converts test definition file to `run.sh` and then parses its stdout. Results | ||
will be saved to results.{json,csv} by test. e.g. | ||
|
||
/root/output/smoke_9879e7fd-a8b6-472d-b266-a20b05d52ed1/result.csv | ||
|
||
When using the same output directory for multiple tests, test-runner combines results | ||
from all tests and save them to `${OUTPUT}/results.{json,csv}`. e.g. | ||
|
||
/root/output/result.json | ||
|
||
## Contributing | ||
|
||
Please use Github for pull requests: https://github.com/Linaro/test-definitions/pulls | ||
|
||
https://git.linaro.org/qa/test-definitions.git is a read-only mirror. New changes in the | ||
github repo will be pushed to the mirror every 10 minutes. | ||
|
||
Refer to [test writing guidelines](automated/doc/test-writing-guidelines.rst) to modify | ||
or add test. | ||
|
||
Changes need to be able to pass sanity check, which by default checks files in the most | ||
recent commit: | ||
|
||
./sanity-check.sh | ||
|
||
To develop locally, there are Dockerfiles in test/ that can be used to simulate | ||
target environments. The easiest way to use is to run `test.sh | ||
[debian|centos]`. test.sh will run validate.py, and then build the Docker | ||
environment specified, run plans/linux-example.yaml, and then drop into a bash | ||
shell inside the container so that things like /root/output can be inspected. | ||
It is not (yet) a pass/fail test; merely a development helper and validation | ||
environment. |
This file was deleted.
Oops, something went wrong.