List of tests

[qarlosalberto]

Hi!
Is there a way to list the tests? Similar to VUnit:
https://vunit.github.io/python_interface.html

get_tests(pattern='*')

    Get a list of tests
    Parameters
        pattern – A wildcard pattern matching the test name
    Returns
        A list of Test objects

[themperek]

Something like this: https://docs.pytest.org/en/latest/example/special.html ?

If use pytest can use all stand procedures

[themperek]

BTW: What is the use case for this?

[qarlosalberto]

We are working in a cocotb-test front-end for TerosHDL:

https://github.com/TerosTechnology/triel

We would like to get two things that we already have implemented with VUnit:

• Test selection: the user has the possibility to list the tests and choose which test to run.

• Independent output by simulator (.vcd): the user can open the waveform of each test. Un ejemplo sería este:

def test_adder_vhdl():
    tests = list("test_adder")
    for x in tests:
        run(vhdl_sources=["../../../hdl/adder.vhd"],
            simulation_args=["--vcd=func"+x[name]+".vcd"],
            toplevel="adder",
            module="test_adder",
            test=x["name"]
            toplevel_lang="vhdl"
        )

[themperek]

There is no explicit list of test it is handled by pytest and there are ways to get it (see: pytest-dev/pytest#1613) some IDE are doing this.

Verilog simulator typically does not take VCD output as a parameter it can be defined in Verilog file itself. As far as I can see one would need to do this via define or similar.

[qarlosalberto]

I think that I can't list the cocotb tests with pytest. In this example:

https://github.com/qarlosalberto/cocotbExamples/blob/master/tools/cocotb/simple/cocotest_test_adder.py
https://github.com/qarlosalberto/cocotbExamples/blob/master/tools/cocotb/simple/test_adder.py

I can discover: def test_adder_vhdl()
But I cant discover: def adder_basic_test(dut), def adder_basic2_test(dut), def adder_random_test(dut

I would like to be able to define with cocotb-test a different output folder (SIM_BUILD) for each cocotb test (adder_basic_test,adder_basic2_test,adder_random_test).

It would also be nice if I could select the tests (TESTCASE)

[themperek]

You can set testcase and sim_build now.

For the list of tests you can try to do something like below and parse std (looks bit ugly):

import py                                                                                                                                                                                                                                                                                                             
import pytest

capture = py.io.StdCapture() 
pytest.main(['--collect-only', '-q']) 
std, err = capture.reset() 
print(std.split("\n")) 

[qarlosalberto]

Ok! I will try, thank you!

[themperek]

Did this work?

[themperek]

I will close. Feel free to re-open if needed.