Improvements: Test Control

[strichmo]

Use this issue to gather all ideas to improve or highlight long-term issues with any of the following:

• Makefiles
• Directory structure
• Simulation control
• Test/regression definitions

Add comments for all ideas to this issue. Please try to refrain from deeper discussion of inidividual items. A separate issue should be opened to enable and record that discussion.

[strichmo]

Test relocatable results directory (e.g. _results) across all common Makefiles and targets)

[strichmo]

Add configuration (CFG=) as a path in all output artifacts (e.g. _results//)

[strichmo]

Consider making <RUN_INDEX> permanent part of all tests. In other words, any test could be run for multiple iterations with different seeds without requiring the _<RUN_INDEX> in the test.yaml file. For default simulations, the RUN_INDEX is 0. For example running the standard hello-world would put outputs in _results/hello-world_0

[MikeOpenHWGroup]

Here's another one: make the test-program generator (e.g. corev-dv, force-riscv) and toolchain components of the UVM environment. This would have several benefits:

• simply the compile and run-flow into a single step.
• make program generation and/or toolchain aware of the Core configuration inputs (boot_addr_i, 'mtvec_addr_i`, etc.) and thus controllable by a UVM testcase.
• state-aware instruction generation.

[MikeOpenHWGroup]

This Issue is really a discussion (which has gone stale) and has, for the most part, been implemented in the CORE-V-VERIF scriptware. I am going to close it.

Watch for a Discussion thread on this topic.