Test Scanner is filename case sensitive

[joshrsmith]

I thought that based on previous discussion, including the runner_cfg generic would be enough for VUnit to pickup a testbench to run tests on. It appears that both the entity and filename of the testbench must also meet the following rules:

def tb_filter(entity):
    """
    Filters entities with both file name and entity name matching tb_* or *_tb.
    """
    # Above docstring can show up in ui.py warnings
    file_ok = basename(entity.file_name).startswith("tb_") or splitext(basename(entity.file_name))[0].endswith("_tb")
    entity_ok = entity.name.startswith("tb_") or entity.name.endswith("_tb")

    if file_ok and entity_ok:
        return True
    return False

This was a problem for me because a testbench was named myblock_TB.vhd instead of myblock_tb.vhd.

I suggest that the scanner allow any case when filtering on tb_* or *_tb.

It might be worth considering bypassing the naming rules if the entity contains the runner_cfg generic.

Regardless, a note should also be added to the documentation that the testbench files and entities must be named in a specific way in order for the scanner to pick them up.

[kraigher]

I think we should change the rule to be based on the prescence of runner_cfg. The old rule was created before there was any runner_cfg.

[joshrsmith]

A rule based on runner_cfg makes sense, if it is required for proper operation of VUnit. If a testbench will work without runner_cfg, then perhaps the existing rule should be updated.

I was surprised after my discussion about #130 when legacy testbenches were scanned/discovered by VUnit when they did not have runner_cfg at all (but they did match current entity/file naming rules).

[kraigher]

I will base the rule entirely on the availability of runner_cfg generic or parameter. I will also give a warning if a module or entity has a runner_cfg without being named tb_* or *_tb case insensitive. A warning will also be given if a test bench is named according to the pattern but does not have a runner_cfg. I think the warning is good to encourage some consistency as well as catch unintendend mistakes.

@cmarqu This will affect the use of test benches where the tests are not at the top level. Your nested module with the TEST_SUITE will be detected to have a runner_cfg. Therefore I will add a NESTED_TEST_SUITE macro that you can use that instead creates a nested_runner_cfg parameter.

Sounds good?

[cmarqu]

Sounds good to me, yes.