Square brackets in test case name prevent ctest from running the test

[pinam45]

Describe the bug

If a test case contain square brackets ([ and ]) in its name, then to call it with the executable you need to escape the brackets but ctest doesn't and the executable return 0 if no test were run.
So ctest call the test with the good name but without escaping the brackets, it doesn't run the test, return 0, ctest display success and you just think the test succeed (very tricky).

For me the problem is that is is not clear that there is a problem running the test, ctest just output that the test succeeded. If it is not possible to fix this problem from Catch2, I think it is necessary to at least warn about this in the documentation.

Expected behavior

Multiple possibilities:

• Make the executable list test with correct call name (with escaped special characters) so the test can be called with a correct name by ctest.
• Make the executable return something different from 0 when no test where run so it become clear that there is a problem when running ctest.
• The documentation say: test name and section name are free form, quoted, strings, maybe forbid square brackets in test case names or just warn about the problem. (If it is already written in the documentation maybe it should be more clear because I couldn't find it.)

Maybe it is more a ctest problem, I don't know if there is this kind of problems with other testing frameworks.

Reproduction steps

Example with my repository, just remane the template test case "array subscript operator" to its old name "operator[]".

Manual test run:

$ ./dynamic_bitset_tests "operator[] - uint16_t" 
===============================================================================
No tests ran

It is clear, the test was not run. Easy to fix, escape the square brackets:

$ ./dynamic_bitset_tests "operator\[\] - uint16_t"
===============================================================================
All tests passed (11972 assertions in 1 test case)

Now let's use ctest:

$ ctest
...
      Start 42: count - uint64_t
42/81 Test #42: count - uint64_t .......................................   Passed    0.03 sec
      Start 43: operator[] - uint16_t
43/81 Test #43: operator[] - uint16_t ..................................   Passed    0.03 sec
      Start 44: operator[] - uint32_t
44/81 Test #44: operator[] - uint32_t ..................................   Passed    0.02 sec
      Start 45: operator[] - uint64_t
45/81 Test #45: operator[] - uint64_t ..................................   Passed    0.02 sec
      Start 46: size num_blocks empty capacity - uint16_t
46/81 Test #46: size num_blocks empty capacity - uint16_t ..............   Passed    0.03 sec
...

The test wasn't run but it is not indicated. I found it when adding code coverage, the code is not executed.

Platform information:

• OS: Debian GNU/Linux buster/sid
• Compiler+version: gcc (Debian 8.3.0-7) 8.3.0
• ctest version: 3.13.4
• Catch version: v2.7.2 (7c37501)

[sfranzen]

You are right. Escaping brackets (and commas) is mentioned in the command line documentation, but the current CatchAddTests CMake module does this only for commas. I'll file a PR with a fix shortly.