Skip to content

Commit

Permalink
Deploy onnx/onnx-mlir to github.com/onnx/onnx-mlir.git:gh-pages
Browse files Browse the repository at this point in the history
  • Loading branch information
traviscibot committed Jan 13, 2021
1 parent d61db06 commit 60458bd
Showing 1 changed file with 11 additions and 4 deletions.
15 changes: 11 additions & 4 deletions Testing.md
Expand Up @@ -12,16 +12,22 @@ cmake --build . --config Release --target check-onnx-backend
```
Packages, such as third_party/onnx and ssl, needs to be installed to run the backend test.

The node tests in onnx that will be run by check-onnx-backend is defined by variable test_to_enable in test/backend/test.py. User can test one test case by environment variable BACKEND_TEST. For example,
The node tests in onnx that will be run by check-onnx-backend is defined by variable test_to_enable in test/backend/test.py. User can test one test case by environment variable TEST_CASE_BY_USER. For example,
```
BACKEND_TEST=selected_test_name cmake --build . --config Release --target check-onnx-backend
TEST_CASE_BY_USER=selected_test_name cmake --build . --config Release --target check-onnx-backend
```
With BACKEND_TEST specified, the intermedia result, the .onnx file and .so file, are kept in build/test/backend for debugging.
With TEST_CASE_BY_USER specified, the intermediate result, the .onnx file and .so file, are kept in build/test/backend for debugging.

When the ONNX-to-Krnl conversion of an operator is added, the corresponding backend tests for this operator should be added to test.py. The available test cases can be found in third_part/onnx/onnx/backend/test/case/node. Please note to add suffix `_cpu` to the onnx test name.

### Tests with unknown dimensions
The onnx node tests usually have known dimension size for input tensors. To test tensor with unknown dimension, the model importer (Build/FrontendONNXTransformer.cpp) provides a functionality to generate such cases. When the environment variable, `IMPORTER_FORCE_DYNAMIC`, is set, the frontend import will turn the all the dimensions (by default) of all the input tensors of the model into -1. For example,

Testing with dynamic tensor sizes is most easily performed by using the following command, also used by our checkers.
```
cmake --build . --config Release --target check-onnx-backend-dynamic
```

The onnx node tests usually have known dimension size for input tensors. So, to test tensor with unknown dimension, the model importer (Build/FrontendONNXTransformer.cpp) provides a functionality to generate such cases. When the environment variable, `IMPORTER_FORCE_DYNAMIC`, is set, the frontend import will turn the all the dimensions (by default) of all the input tensors of the model into -1. For example,
```
IMPORTER_FORCE_DYNAMIC='-1:-1' all dimensions of all the inputs will be changed
IMPORTER_FORCE_DYNAMIC='0:-1' all dimensions of the first input will be changed
Expand Down Expand Up @@ -60,6 +66,7 @@ with IMPORTER_FORCE_DYNAMIC='0:0,2|1:1', the result is:

This is a way to use existing node test for dynamic tensors. Since not all test case can pass with dynamic tensor, there is a list in test/backend/test.py, test_not_for_dynamic, to specify which test can not pass with IMPORTER_FORCE_DYNAMIC is defined.


## LLVM FileCheck Tests

TODO.
Expand Down

0 comments on commit 60458bd

Please sign in to comment.