[UnitTests][Contrib] Enable contrib tensorrt/coreml unit tests #8902
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The unit tests in `test_coreml_codegen.py` haven't run in the CI
lately, so this test wasn't caught before. (See tracking issue
- Added `pytest.mark.xfail` mark to `test_annotate`.
- Added `tvm.testing.requires_package` decorator, which can mark tests
as requiring a specific python package to be available. Switched
from `pytest.importorskip('coremltools')` to
`requires_package('coremltools')` in `test_coreml_codegen.py` so
that all tests would explicitly show up as skipped in the report.
- Added `uses_gpu` tag to all tests in `test_coreml_codegen.py`, since
only ci_gpu has coremltools installed. In the future, if the ci_cpu
image has coremltools installed, this mark can be removed.
…cuda Previously, the tests had an early bailout if tensorrt was disabled, or if there was no cuda device present. However, the tests were not marked with `pytest.mark.gpu` and so they didn't run during `task_python_integration_gpuonly.sh`. This commit adds the `requires_cuda` mark, and maintains the same behavior of testing the tensorrt compilation steps if compilation is enabled, and running the results if tensorrt is enabled. In addition, some of the tests result in failures when run. These have been marked with `pytest.mark.xfail`, and are being tracked in issue apache#8901.
Lunderberg
requested review from
areusch,
comaniac,
jroesch,
junrushao,
merrymercy,
tqchen,
yzhliu and
zhiics
as code owners
tkonolige
approved these changes
Sep 1, 2021
leandron
approved these changes
Sep 2, 2021
AndrewZhaoLuo
added a commit
to AndrewZhaoLuo/tvm
that referenced
this pull request
Sep 2, 2021
* main: [UnitTests][Contrib] Enable contrib tensorrt/coreml unit tests (apache#8902) [BUG] DataType Bug In SplitRel (apache#8899) Enable python debug runtime for exported network libraries (apache#8793) Set default value of p in LpPool as 2 (apache#8866) [Community] @Hzfengsy -> Committer (apache#8908) Trivial uTVM -> microTVM "spelling" fix to align with branding. (apache#8905) [Vulkan][Topi] Parametrizing additional topi tests, marking vulkan failures (apache#8904) Move to new style issue template system (apache#8898) [Onnx] Support Negative Log Loss (apache#8872) [ROCm][TVMC] Add ROCm to the TVMC driver (apache#8896) fix error report on Store (apache#8895) [Docker] Re-enabled automatic --tty flag when running bash. (apache#8861)
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Sep 29, 2021
…e#8902) * [UnitTests][CoreML] Marked test_annotate as a known failure. The unit tests in `test_coreml_codegen.py` haven't run in the CI lately, so this test wasn't caught before. (See tracking issue - Added `pytest.mark.xfail` mark to `test_annotate`. - Added `tvm.testing.requires_package` decorator, which can mark tests as requiring a specific python package to be available. Switched from `pytest.importorskip('coremltools')` to `requires_package('coremltools')` in `test_coreml_codegen.py` so that all tests would explicitly show up as skipped in the report. - Added `uses_gpu` tag to all tests in `test_coreml_codegen.py`, since only ci_gpu has coremltools installed. In the future, if the ci_cpu image has coremltools installed, this mark can be removed. * [Pytest][TensorRT] Mark the TensorRT tests with tvm.testing.requires_cuda Previously, the tests had an early bailout if tensorrt was disabled, or if there was no cuda device present. However, the tests were not marked with `pytest.mark.gpu` and so they didn't run during `task_python_integration_gpuonly.sh`. This commit adds the `requires_cuda` mark, and maintains the same behavior of testing the tensorrt compilation steps if compilation is enabled, and running the results if tensorrt is enabled. In addition, some of the tests result in failures when run. These have been marked with `pytest.mark.xfail`, and are being tracked in issue apache#8901.
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 13, 2022
…e#8902) * [UnitTests][CoreML] Marked test_annotate as a known failure. The unit tests in `test_coreml_codegen.py` haven't run in the CI lately, so this test wasn't caught before. (See tracking issue - Added `pytest.mark.xfail` mark to `test_annotate`. - Added `tvm.testing.requires_package` decorator, which can mark tests as requiring a specific python package to be available. Switched from `pytest.importorskip('coremltools')` to `requires_package('coremltools')` in `test_coreml_codegen.py` so that all tests would explicitly show up as skipped in the report. - Added `uses_gpu` tag to all tests in `test_coreml_codegen.py`, since only ci_gpu has coremltools installed. In the future, if the ci_cpu image has coremltools installed, this mark can be removed. * [Pytest][TensorRT] Mark the TensorRT tests with tvm.testing.requires_cuda Previously, the tests had an early bailout if tensorrt was disabled, or if there was no cuda device present. However, the tests were not marked with `pytest.mark.gpu` and so they didn't run during `task_python_integration_gpuonly.sh`. This commit adds the `requires_cuda` mark, and maintains the same behavior of testing the tensorrt compilation steps if compilation is enabled, and running the results if tensorrt is enabled. In addition, some of the tests result in failures when run. These have been marked with `pytest.mark.xfail`, and are being tracked in issue apache#8901.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some unit tests in the contrib directory have not been running in CI (see #8901 for tracking issue). This PR re-enables these tests, and marks any regressions with
xfail.[Pytest][TensorRT] Mark the TensorRT tests with tvm.testing.requires_cuda
Previously, the tests had an early bailout if tensorrt was disabled, or if there was no cuda device present. However, the tests were not marked with
pytest.mark.gpuand so they didn't run duringtask_python_integration_gpuonly.sh. This commit adds therequires_cudamark, and maintains the same behavior of testing the tensorrt compilation steps if compilation is enabled, and running the results if tensorrt is enabled.In addition, some of the tests result in failures when run. These have been marked with
pytest.mark.xfail, and are being tracked in issue [Tracking][Contrib] Known failing unit tests #8901.[UnitTests][CoreML] Marked test_annotate as a known failure.
The unit tests in
test_coreml_codegen.pyhaven't run in the CI lately, so this test wasn't caught before. (See tracking issue #8901).Added
pytest.mark.xfailmark totest_annotate.Added
tvm.testing.requires_packagedecorator, which can mark tests as requiring a specific python package to be available. Switched frompytest.importorskip('coremltools')torequires_package('coremltools')intest_coreml_codegen.pyso that all tests would explicitly show up as skipped in the report.Added
uses_gputag to all tests intest_coreml_codegen.py, since only ci_gpu has coremltools installed. In the future, if the ci_cpu image has coremltools installed, this mark can be removed.