mesontest: Improve test suite selection.

[trhd]

Continue discussion from #1252. Could also fix #1273.

Apparently mesontest's suite option already accepted arguments of the format "$subproject.$suite" so the idea was to extend this functionality.

With this change:

• --suite option can be given multiple times.
• Also "$main_project.$suite" is accepted.
• Any project name can be used as such as a suite selector.
• Any suite name can be used to match tests from any project if prefixed with a period ('.').
• The output with the --list option matches the output from when it is not given (w.r.t. the test project, suite and name combination).

Some examples:

• All tests in $main_project: --suite $main_project
• All tests in $subproject: --suite $subproject
• All tests in $main_project part of suite 'foobar': --suite $main_project.foobar
• All tests part of a suite called 'foobar' (in any project): --suite .foobar
• All tests in $subproject's suite 'barfoo' and all tests in $subproject2: --suite $subproject.barfoo --suite $subproject2
• All tests with the name 'supertest' or 'shittytest' in $subproject's suite 'feaX': --suite $subproject.feaX supertest shittytest

Output will look like: $project.$suite / $test_name or just $test_name if all the tests belong to the same suite (and project).

[msink]

How to exclude some tests?

Maybe something like this: if name prefixed with the ~ character then it matches an exclusion.

[trhd]

Updated this. Now one can use ~$suite to exclude test suites. For example, to include all tests from a (sub)project myprj except those included in a suite called long:

• --suite myprj --suite ~.long

[jpakkane]

The naming thing requires some thought. Project options are set with subprojname:option. It would be nice if we had only one way of namespacing subprojects and used that consistently everywhere.

[trhd]

I was torn between . and : as well. Eventually chose . as $subproje.$suite was already supported. Quite simple to change though, should we so choose.

Also, .somesuite requires a key press less than :somesuite. On the other hand, :somesuite looks (at least in my opinion) cooler than .somesuite.

[msink]

.somesuite - name starting with a period - can be used to specify hidden by default tests, like in posix filenames.
So imho : or / as divider is better.

[jpakkane]

--suite myprj --suite ~.long

Let's not do magic ascii art, but instead something like --remove-suite log.

[trhd]

Changed the . divider to : and replaced the ~ negation sign with a new option --no-suite.

E.g. --suite myprj --no-suite :long.

[jpakkane]

This really needs a test. Please add one in run_unittests.py that does the following:

• has two tests, one of which fails, put them in suites ok and fail
• run basic test, check that it fails
• run test with --no-suite fail, check that it works
• run test with --no-suite ok, check that it fails

Thanks,

[trhd]

Overdid it.