Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make astrolabe resilient to faulty specification test files #7

Closed
prashantmital opened this issue Feb 28, 2020 · 1 comment
Closed
Labels
bug Something isn't working

Comments

@prashantmital
Copy link
Contributor

prashantmital commented Feb 28, 2020

Consider a situation wherein there exists a typo in one of the test scenario files - say an expected field is missing. As written currently, the test runner will notice the error when it attempts to access the missing field, probably in astrolabe.spec_runner.AtlasTestCase.run. Attempting to access this non-existent field would raise an AttributeError which would be propagated back to the user and terminate the entire test run.

astrolabe should be robustified to deal with this kind of erroneous test file content. A faulty spec test should only cause that particular test to fail, but should not impede the remainder of the test run from proceeding as usual.

@prashantmital prashantmital added the bug Something isn't working label Feb 28, 2020
@prashantmital
Copy link
Contributor Author

UPDATE: since we now run one-test-per-task a faulty test scenario file will no longer impede other tests (for which the files don't have any errors). Consequently, this is no longer a major concern and can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant