-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[hexagon][testing] add test-skip logic; fixes #11737
Conversation
a755f68
to
da9ac8a
Compare
Updated commit message: [hexagon][testing] add test-skip logic; fixes
|
tests/python/contrib/test_hexagon/test_benchmark_elemwise_add.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cconvey Please fix the CI.
52a43e5
to
52e26ef
Compare
- Skip Hexagon benchmarks whenever the env. var `ANDROID_SERIAL_NUMBER` has the value `simulator`. This is a temporary hack to prevent the CI pre-commit hook from running benchmarks, due to the extra time required. - Fix a bug where the elementwise-add benchmark code was broken by an earlier change to the `HexagonLauncherRPC` class. - Rename `benchmark_elemwise_add.py` to `test_benchmark_elemwise_add.py` so that it's noticed by the CI test infrastructure. (CI tests are sometimes run in contexts _other than_ the pre-commit hook.) - Miscellaneous small changes to `tests/python/contrib/test_hexagon/benchmark_util.py`.
- Skip Hexagon benchmarks whenever the env. var `ANDROID_SERIAL_NUMBER` has the value `simulator`. This is a temporary hack to prevent the CI pre-commit hook from running benchmarks, due to the extra time required. - Fix a bug where the elementwise-add benchmark code was broken by an earlier change to the `HexagonLauncherRPC` class. - Rename `benchmark_elemwise_add.py` to `test_benchmark_elemwise_add.py` so that it's noticed by the CI test infrastructure. (CI tests are sometimes run in contexts _other than_ the pre-commit hook.) - Miscellaneous small changes to `tests/python/contrib/test_hexagon/benchmark_util.py`.
Make the execution of benchmarks in
benchmark_elemwise_add.py
dependon two environment variables:
ANDROID_SERIAL_NUMBER
andTVM_FORCE_RUN_BENCHMARKS
.By default, the benchmarks are skipped when the
ANDROID_SERIAL_NUMBER
environment variable has the valuesimulator
. (Case is significant.)This can be overridden by setting the environment variable
TVM_FORCE_RUN_BENCHMARKS
to the value1
.Maintainer note: This is a (hopefully) temporary measure to prevent
(time-consuming) benchmarking code from executing every time TVM's
pre-commit CI hook runs.
Fix a bug where
benchmark_elemwise_add.py
was broken by anearlier change to the
HexagonLauncherRPC
class.misc. small changes to
tests/python/contrib/test_hexagon/benchmark_util.py
.cc @mehrdadh