Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some test modules are using unittest.main() instead of common_utils.run_tests() #50448

Closed
xwang233 opened this issue Jan 12, 2021 · 2 comments
Closed
Assignees
Labels
better-engineering Relatively self-contained tasks for better engineering contributors enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: testing Issues related to the torch.testing module (not tests) module: tests Issues related to tests (not the torch.testing module) triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@xwang233
Copy link
Collaborator

xwang233 commented Jan 12, 2021

🚀 Feature

Some test modules are using unittest.main() instead of torch.testing._internal.common_utils.run_tests(). Thus, the arg parser in common_utils are not running for these tests, which makes the whole test suite somewhat inconsistent.

parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('--subprocess', action='store_true',
help='whether to run each test in a subprocess')
parser.add_argument('--seed', type=int, default=1234)
parser.add_argument('--accept', action='store_true')
parser.add_argument('--jit_executor', type=str)
parser.add_argument('--repeat', type=int, default=1)
parser.add_argument('--test_bailouts', action='store_true')
parser.add_argument('--save-xml', nargs='?', type=str,
const=_get_test_report_path(),
default=_get_test_report_path() if bool(os.environ.get('IN_CI')) else None)
parser.add_argument('--discover-tests', action='store_true')
parser.add_argument('--log-suffix', type=str, default="")
parser.add_argument('--run-parallel', type=int, default=1)

test_expecttest
test_mobile_optimizer
test_show_pickle
test_namedtuple_return_api 
test_jit_disabled 
test_overrides 
test_tensorexpr 
test_package 

We encountered this in a problem while running tests

python test/run_test.py --verbose -- --save-xml -v

--save-xml should be passed to each test module, but some test modules not using common_utils.run_tests() don't have these parser and will just throw an error: unrecognized arguments: --save-xml

cc @mruberry @VitalyFedyunin @walterddr @ptrblck

@izdeby izdeby added better-engineering Relatively self-contained tasks for better engineering contributors enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: testing Issues related to the torch.testing module (not tests) module: tests Issues related to tests (not the torch.testing module) triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jan 12, 2021
@mruberry
Copy link
Collaborator

cc @malfet

@walterddr
Copy link
Contributor

walterddr commented Jan 13, 2021

looks like this is in fact the case. also in circle it doesn't actually show unrecognized arguments, instead it simply just missed the "Generating XML reports..." logs. I will take a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
better-engineering Relatively self-contained tasks for better engineering contributors enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: testing Issues related to the torch.testing module (not tests) module: tests Issues related to tests (not the torch.testing module) triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants