Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support merging covdbs with different top-level architectures #823

Open
sean-anderson-seco opened this issue Jan 5, 2024 · 3 comments
Open
Labels

Comments

@sean-anderson-seco
Copy link
Contributor

sean-anderson-seco commented Jan 5, 2024

At the moment, it is only possible to merge covdbs which share the same top-level architecture. Merging coverage databases with different top-level architectures results in only the coverage from the first database being kept.

However, it is common to instantiate architectures in a hierarchy. This can result in misleading coverage results. For example, it could appear that A.B did not have sufficient coverage, even though there is a separate test for B which includes paths not covered in the test of A.B.

We should add support for merging covdbs with different top-level architectures, allowing a coverage summary to be created and easing identification of areas not covered under any test.

@nickg nickg added the coverage label Jan 5, 2024
@Blebowski
Copy link
Sponsor Contributor

Hi,

it always depends on what you are intending to cover. Sometimes, it is desirable to target every item for the given hierarchy.
If you have single entity instantiated multiple times (e.g. a memory instantiated multiple times via generate), you
would typically want to cover access to each such memory, so merging all the data for the memory entity from all its
instantiations will actually hide some of the uncovered data. When you have a TB with regression that achieves a coverage
for each such instantiation, you can be more-less sure, that once you replace your DUT with gate-level netlist, you will
catch cases where synthesizer optimized away part of your design. In other words, you check how much is your
"logic" covered.

Commercial tools support merging the coverage for each source file (instead of by hierarchy/logic). It would be
definitely good to add this type of coverage too. The report would then contain list of source files, instead of hierarchy.

What you are reffering to, is somehow mixture of both approaches. To have "per-hierarchy" coverage, but with option
to "add coverage" to a sub-block from a different TB that instantiates the same sub-block. E.g. if you have a top level
TB for an IP, but the IP has a sub-block whose logic is very hard to cover in the top level TB. So, you create a unit-test
for such sub-block, and you would like to have the coverage data for the sub-block from this unit test merged into the
hierarchy coverage of whole IP from the top-level TB.

I would not do that somehow automatically, otherwise there is the aliasing/hiding danger as described above. But,
I would maybe extend the -c command with an option like:

--fold=<entity_name>

Such option would "fold" together all the coverage that is under the entity_name block. Should it fold together
also sub-entities of <entity_name> ? I would say yes...

So, to sum it up, I think two things would be good:

  1. A "per source file" merging mode (as oppose to "per hierarchy" mode).
  2. Option to "fold" together coverage for some entity from different hierarchies when operating in the "per hierarchy" mode.

@sean-anderson-seco
Copy link
Contributor Author

Commercial tools support merging the coverage for each source file (instead of by hierarchy/logic). It would be
definitely good to add this type of coverage too. The report would then contain list of source files, instead of hierarchy.
...
So, to sum it up, I think two things would be good:

  1. A "per source file" merging mode (as oppose to "per hierarchy" mode).
  2. Option to "fold" together coverage for some entity from different hierarchies when operating in the "per hierarchy" mode.

Either of these would be fine for my use-case.

I think the existing coverage is good for developing/verifying testbenches, but it is hard to get a high-level overview for a whole project.

@Blebowski
Copy link
Sponsor Contributor

Current state:

  1. Merging of two different top hierarchies
  2. Per-file merging mode
  3. Folding of coverage with different hierarchy, but the same entity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants