veriti is a simulation based verification framework for digital hardware designs.
The framework can be divided into 3 major actions:
- Model the hardware design in software to generate input vectors and expected output vectors.
- Simulate the hardware design by sending input vectors to the design under test and logging comparisons between simulated output vectors and expected output vectors.
- Analyze the logged events for any errors.
Run the following Pip command:
pip install git+https://github.com/cdotrus/veriti.git@0.1.0#egg=veriti
Run the following Orbit command:
orbit install veriti:0.1.0 --url "https://github.com/cdotrus/veriti/archive/refs/tags/0.1.0.zip"
Add the following line to your Orbit.toml:
veriti = "0.1.0"
Verification is an important process in the hardware development cycle that ensures a design functions as intended and is free of bugs. As a design increases in complexity, the number of coding errors also increases. Therefore, one approach to verifying a logic design conforms to specification is functional verification, where tests are performed to check if the design does what is intended under a set of conditions. However, designs typically have an insurmountable amount of test cases to create and perform each one. Therefore, producing test cases and determining when enough test cases is "enough" requires a well-designed and efficient approach.
Any hardware design can be translated behaviorally to software, and by writing the model in software, it provides an extra level of cross-checking to ensure the hardware design matches specification. Writing the model in software is also typically easier with nicer language constructs and an abundance of existing libraries available. With veriti, all of the plumbing in setting up tests is minimized so testing the next hardware design only requires focusing on writing the model, not the whole framework.
veriti is a collection of functions that help generate, store, load, and analyze test cases. The functions are implemented in both software and hardware description languages (HDL) to facilitate the interaction between the framework's layers of abstraction. These functions allow the user largely focus on only two components when performing functional verification: the software model and the hardware design.
This framework attempts to decouple the functional and timing aspects of a hardware simulation. The functional model is written in software, while the exact timing of how to monitor and check the design under test is kept in HDL. This separation of layers allows each language to excel at what it is good at.
The following objectives drive the design choices behind building this framework:
- ease of use: Verifying the next design should be easy to set up and configure
- general-purpose: Be generic and allow the user enough control to support a wide range of designs, from purely combinational logic to control-flow architectures
- increased productivity: Using the framework should result in shorter times spent in the verification phase due to reusing highly modular components and insightful results
The veriti framework is divided into three main layers.
- Software Layer: low-level functions to generate inputs and outputs and analyze recorded data
- Data Layer: persistent storage of data to be shared between hardware and software layers
- Hardware Layer: low-level functions to load inputs and outputs, drive inputs, check outputs, and log events
This separation of functionality is important for modularity. If a model needs to be written in a different language (Python/C++/Rust), then only the software layer requires changes; the data layer and hardware layer are left unmodified. Having well-defined interfaces between these layers allows for the framework to easily expand to new software languages and HDLs.
The software layer implements the low-level functions required to run any form of test. It translates your test cases into the data layer represented by a specific file format.
The software layer is responsible for generating test inputs, tracking coverage, and generating test outputs. When defining signals in your software model, you can also specify their probability distribution to randomly sample based on distributions. If not specified, the default is uniform distribution.
The software layer can also generate HDL code, which can be directly copied into the testbench for establishing connections between the hardware design and the data layer.
The software layer is available as a library and as a stand-alone program.
The data layer stores the tests to run during simulation and the expected outputs. This information is typically stored in a specific file format already handled by veriti.
Each line in a data file is a transaction. A transaction in this sense is the combination of complete set of inputs or outputs. For data stored in an input file, each transaction is to be the input into the design-under-test on a single clock cycle. For data stored in an output file, each transaction is the outputs to be checked against the design-under-test's outputs in the scoreboard. The output transactions do not have to be checked every clock cycle, and may only be cared when a certain condition occurs (such as a valid signal being asserted).
The number of transactions stored as inputs and outputs does not have to be 1-to-1. There may be more input transactions (fed every clock cycle) than output transactions (only checked when valid).
The hardware drivers implement the low-level functions required to receive data from the data layer. This data is read during simulation to run test cases and automatically assert outputs.
The hardware layer is responsible for the timing of the simulation: specifically determining when to drive inputs and monitoring when to check outputs.
- Fine-grain control over when to send inputs and check outputs, produce inputs or outputs cycle-by-cycle or wait on particular control signals
- Ability to enable coverage-driven test generation (CDTG) to help minimize the number of tests required to achieve the target coverage
- Supported coverage nets:
CoverPoint,CoverRange,CoverGroup,CoverCross - Ability to generate HDL glue-logic code per design-under-test to connect hardware drivers layer to the data layer
Verification is done through simulation at the hardware level. The hardware simulation is trace-based; the set of inputs and outputs are pre-recorded before the simulation begins. These traces are stored in the data layer.
The workflow is broken down into 3 main steps:
-
Run the software model using
veritisoftware drivers to write files at the data layer for design under test's inputs and expected outputs based on defined coverage. -
Run hardware simulation to read/parse inputs and outputs and record outcomes into a log file using
veritihardware drivers. -
Run a software program (
veriti) to interpret/analyze outcomes stored in log file. If all tests passed, then the program exits with code0. If any tests failed, then the program exits with code101.
When the software model is generating tests, it can also keep track of what test cases are being covered by using coverage nets, such as CoverGroups or CoverPoints. By handling coverages in software, it allows for coverage-driven test generation (CDTG) by choosing the next set of inputs that will work toward achieving total coverage.
Once the test files are generated at the data layer, the simulation can begin in the hardware description language. At the hardware drivers layer, a package of functions exist for clock generation, system reseting, signal driving, signal montioring, and logging.
- cocotb: coroutine based cosimulation testbench environment for verifying VHDL and SystemVerilog RTL using Python