Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

do not exclude travis markers in plugin test runner #15

Open
mschrimpf opened this issue Nov 16, 2022 · 2 comments
Open

do not exclude travis markers in plugin test runner #15

mschrimpf opened this issue Nov 16, 2022 · 2 comments

Comments

@mschrimpf
Copy link
Member

mschrimpf commented Nov 16, 2022

the plugin runner currently excludes all tests that cannot be run on travis: https://github.com/brain-score/language/blob/dac042286911a0f42d897982fcf9ac45d3775c18/brainscore_language/plugin_management/test_plugin.sh#L25

This exclusion should only be set for travis runs (e.g. in .travis.yml) so that such tests in general are not excluded, e.g. for execution on openmind.

@kvfairchild
Copy link
Contributor

Should we not exclude any tests as a default? Or should we have some default exclusions (e.g. not requires_gpu) and override as needed?

@mschrimpf
Copy link
Member Author

I think the running system should set these exclusions and ideally be as non-restrictive as possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants