Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testament: use spec tags (more flexible than file hierarchies / dir category) #445

Open
timotheecour opened this issue Dec 11, 2020 · 0 comments

Comments

@timotheecour
Copy link
Owner

timotheecour commented Dec 11, 2020

File hierarchies don't convey fact that a given file (eg tests/untestable/thttpclient_ssl_disabled.nim) can be associated with different categories, in this case: integration, ssl, networking, etc.

Instead it's much better to rely on tags, and testament spec is a natural fit for that, eg:

discard """
  targets: "c cpp"
  tags: "integration, ssl, benchmark, networking"
"""

benefits

  • This also allows adding/removing/editing tags for a file without having to move files around and loose much of git history in the process.

  • Tags would also offer better filtering, eg allowing one to run all tests with tags js and benchmark but not networking: testament --tags:"js,benchmark,-networking"

example tags

  • networking
  • benchmarking
  • integration
  • flaky
  • windows
  • ssl
  • parser
  • all existing categories (ie the 1st-level directory under tests, eg ccgbugs for tests/ccgbugs/mymodule.nim) are implicit tags
    etc

notes

  • tags would be checked by testament against a list to avoid mistyped tags

links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant