Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use exclusion list to disable select tests from direct mode testing #1562

Closed
mih opened this issue Jun 12, 2017 · 2 comments · Fixed by #1789
Closed

Use exclusion list to disable select tests from direct mode testing #1562

mih opened this issue Jun 12, 2017 · 2 comments · Fixed by #1789
Assignees
Labels
WIP work in progress

Comments

@mih
Copy link
Member

mih commented Jun 12, 2017

ATM we test a little bit of code for compliance with direct mode, using hand-picked tests. IMHO this should change. There is quite a chunk of usable functionality in direct mode, but from the test perspective it doesn't look like it. I think we should switch to a negative list, disabling only test that fail (possibly for a known reason). The current approach will or will not allow for the status quo to get worse -- we just won't know -- hence it isn't suitable for making progress towards direct mode compatibility.

@bpoldrack
Copy link
Member

I agree. Additionally the same is true for v6.

@bpoldrack bpoldrack added the WIP work in progress label Sep 1, 2017
@bpoldrack
Copy link
Member

bpoldrack commented Sep 1, 2017

This is a bit more of an effort than I expected it to be.
Apparently nose can't easily exclude a list of tests out of the box, but only entire files/directories or via regexes.
Current approach: Have a "dry-run" of nose to discover all tests (sadly by name), do a lot of grep'ing and sed'ing to get a unique list of tests adressed via /path/to/file.py:test, build difference to exclusion list and build a nose.cfg from the result to run those tests only.
The exclusion list itself (actually there are two - DM and V6) is build similarly but in an one time effort and is supposed to be maintained manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
WIP work in progress
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants