You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In upcoming releases (#763), we expect to implement #559 (see #757 and #764), which will introduce some changes to the management of builtins. Builtins are the HDL libraries distributed with VUnit, some of which are required and some are optional. I believe we should take this opportunity to think about the API.
Currently, the list of "modules/libraries" is the following:
data_types
core
logging
string_ops
check
dictionary
run
path
array_util
com
verification_components
random
osvvm (submoduled)
json4vhdl (submoduled)
The internal API provides a helper (named add), and the following explicit functions:
_add_data_types
data_types
_add_array_util
array_util
_add_random
random
_add_com
com
_add_verification_components
verification_components
_add_osvvm
osvvm
_add_json4vhdl
json4vhdl
add_verilog_builtins
vunit_pkg.sv
add_vhdl_builtins
core
logging
string_ops
check
dictionary
run
path
Furthermore:
_add_data_types is executed in _add_vhdl_builtins, and the following dependencies exist:
verification_components depends on com and osvvm.
random depends on osvvm.
All the functions except the external code (osvvm and json4vhdl) use a helper (_add_files), because a file naming convention is used to indicate which standard is supported. That is something we cannot enforce in external projects.
Function osvvm_is_installed requires OSVVM to be installed (the submodule to be checked out) so that VUnit can be used. That is more of an historical remainder than an a sensible solution nowadays.
That was required because of backwards compatibility. Although it is possible to disable add_builtins through option compile_builtins, the default is to compile them (that is, data_types, core, logging, string_ops, check, dictionary, run and path). Therefore, when verilog support was added to VUnit, the existing function was preserved. Moreover, compile_builtins was initially meant as a development feature (for running internal unit tests), and not to be explicitly used by consumers/users of VUnit.
As discussed in #559, it would be better to expose the two explicit add_*_builtins in the UI, which would allow to get rid of the vunit.verilog class. That is being addressed in #757. It does modify the public API (UI), but builtins.py is not modified.
array_util is deprecated since 2y ago, and it will be removed soon. See #600. The _add_array_util function in builtins.py will be preserved, but it will raise an explicit RuntimeError. VHDL sources will be removed.
Other deprecated VHDL sources will be removed (see #763), but those have no impact on builtins.py or the public API.
In this context, the main disruptive change is that #764 will require all current users to use add_vhdl_builtins or add_verilog_builtins explicitly. As a result, most users will have snippets such as the following:
# example array_axis_vcsVU=VUnit.from_argv()
VU.add_vhdl_builtins()
VU.add_verification_components()
# example axi_dmaVU=VUnit.from_argv()
VU.add_vhdl_builtins()
VU.add_osvvm()
VU.add_verification_components()
# example comVU=VUnit.from_argv()
VU.add_vhdl_builtins()
VU.add_com()
VU.add_verification_components()
VU.add_osvvm()
On the one hand, I believe we can do better with regard to verbosity. I'd like to propose two alternatives which are not exclusive:
# example axi_dmaVU=VUnit.from_argv()
VU.add('vhdl_builtins', 'osvvm', 'verification_components')
# example axi_dmaVU=VUnit.from_argv(add=['vhdl_builtins', 'osvvm', 'verification_components'])
NOTE: We might want to discuss whether add is expressive enough.
That is, allow specifying which optional libraries to compile through a list of strings, instead of requiring an explicit function for each of them. Passing a list of strings makes it easier to add/remove libraries in the future, as well as having aliases for them. For instance, we might support vhdl as an alias of vhdl_builtins and vcs as an alias of verification_components. I would also agree with using an enumeration instead of strings, for better type checking.
On the other hand, we might want to split vhdl_builtins into core and utils. I don't mean removing vhdl_builtins, but making it an alias of utils, which depends on core. The resulting list of keywords, packages and dependencies would be the following:
/cc @LarsAsplund@kraigher can you please confirm that this "tree" of dependencies is correct? I would like to add a diagram to the docs explaining this. My main doubts are:
Is it possible to use VUnit without the run package? Why is it not part of core?
Do you have any clear guess about the dependencies between logging/string_ops/check/path and data_types/core/dictionary/run?
Should verification_components be explicitly dependent on random?
We can also take this opportunity to rethink the integration of third-party sources. We are currently using the core of OSVVM and JSON-for-VHDL through git submodules. Those are required because some of the features in VUnit utils or examples depend on them. However, users might want the same ease of addition for the whole OSVVM, for UVVM, SVUnit, or any other public or in-house library. The fact that users can already extend their setup through add_source_files seems to be overseen by multiple users and maintainers of other projects, and there is the unfortunate perception about some sources being first classs citizens.
That was discussed in #511, and it was considered not convenient to embed git/wget/curl into the Python codebase of VUnit. I agree it is desirable to let users download the sources however they want, but I don't think each user should guess or copy and paste the snippet of add_source_files statements.
Therefore, I believe that the approach should be documentation. As far as I am aware, there are two solutions for users to write reusable Python snippets that extend VUnit:
This is probably the most naive approach. It is functional, easy to understand and to use by users without strong software knowledge.
A class extending the VUnit UI. The soon to be deprecated vunit.verilog class showcases how to override a method of the public API. There are three possible non-exclusive approaches:
Override the add method proposed above, or add new add_* methods.
Override the from_argv and from_args methods to process some items from the add argument, before calling the methods of the parent (super?) class.
Override the Builtins class/object. I know this needs to be technically possible, but I'm unsure about the complexity or whether there is an actual use case for it.
As illustrated in OSVB: Overview, currently each verification framework/methodology uses a very different format for defining the sources:
VUnit uses the functions/API explained above.
OSVVM uses .pro files which are TCL code and need to be executed in a given order. Sources are listed in compilation order.
UVVM uses .txt files. Sources are listed in compilation order.
Cocotb uses makefiles.
My initial intent some months ago was to use FuseSoC's .core files for replacing OSVVM's and UVVM's .pro and .txt sources. As a matter of fact, @Paebbels' vendor scripts (https://github.com/ghdl/ghdl/tree/master/scripts/windows) do parse those files already. The scripts are currently written in PowerShell and Bash, but we expect to convert them to Python (see ghdl/ghdl#1821). Hence, we have the knowledge for reading the files, but unfortunately don't have the knowledge for reading FuseSoC's .core (CAPI 2) format. That's because there is no reusable and documented Python API for interacting with CAPI 2, there is no "language model". Therefore, I prototyped pyCAPI as a proof of concept for showcasing how might a FuseSoC-VUnit integration look like, should FuseSoC have a reusable Python API.
After multiple discussions in the community, I/we decided not to waste time in a probably limited declarative format (either reused or invented), and instead use imperative scripts straightaway, i.e. Python scripts. That was one of the triggering factors for creating EDAA. pyEDAA.ProjectModel provides an abstraction that can be used for representing the whole OSVVM, UVVM and/or any other "third-party library". Therefore, writing a single ProjectModel-to-VUnitProject conversion function/class might suffice. OSVVM and UVVM would, on their own, provide Python scripts describing their sources as a ProjectModel, and VUnit would use a generic solution. I'm not sure about ProjectModel-to-VUnitProject being a good fit in the pyEDAA.ProjectModel repo. It should probably be contributed in this repository.
However, OSVVM and UVVM ProjectModel scripts were not written yet, neither the ProjectModel-to-VUnitProject conversion. Hence, it might be more straightforward to skip ProjectModel for now, and use the VUnit API directly. In the end, the main reason to use EDAA's API is because synthesis is out of scope of VUnit; otherwise, the EDAA ProjectModel would not be necessary. I.e., VUnit's solution is already an imperative script based approach using OOP.
My main doubt here is distribution. We don't want to add vunit.osvvm and vunit.uvvm classes in this repo. If we were to do so, we might just add those to the builtins.py, but that's the opposite of our purpose: we want a more generic solution, not double betting on manually supporting each third-party project. I'm thinking I might have those classes in OSVB temporarily (until we decide a better/definitive location). The problem is that OSVB is not published to PyPI, and I would honestly want to prevent users from considering OSVB a product per se. It is a place for discussion and to upstream enhancements, not a product. At the same time, I'm afraid that it can be problematic to force users to install some Python sources from OSVB in order to use the VUnit-OSVVM and VUnit-UVVM integrations.
I think that the first "users" might be developers; the developers of OSVVM and UVVM. If they want to collaborate in this effort, I can provide a GitHub Action, a reference workflow and a run.py script which are ready-to-use, including something such as the following:
The target would be to run three testbenches, at least, one using each framework/methodology.
That will allow all of us to better understand the challenges for reusing VUnit's Python codebase while best integrating with the features provided by OSVVM and/or UVVM. According to GitHub Facts About the HDL Industry: Wilson Study Comparison, it would get 60% of the FPGA market to use the same CI framework, regardless of the preferred testbench methodology and/or HDL utils.
In upcoming releases (#763), we expect to implement #559 (see #757 and #764), which will introduce some changes to the management of builtins. Builtins are the HDL libraries distributed with VUnit, some of which are required and some are optional. I believe we should take this opportunity to think about the API.
NOTE: all the information below is found in vunit/builtins.py.
Currently, the list of "modules/libraries" is the following:
The internal API provides a helper (named
add
), and the following explicit functions:Furthermore:
_add_data_types
is executed in_add_vhdl_builtins
, and the following dependencies exist:_add_files
), because a file naming convention is used to indicate which standard is supported. That is something we cannot enforce in external projects.osvvm_is_installed
requires OSVVM to be installed (the submodule to be checked out) so that VUnit can be used. That is more of an historical remainder than an a sensible solution nowadays.Kind of suprisingly, the public API of VUnit (UI) exposes most of the functions listed above, except
_add_data_types
,add_verilog_builtins
andadd_vhdl_builtins
. Instead, there is a singleadd_builtins
that defaults toadd_vhdl_builtins
and it is overriden in classvunit.verilog
. See https://github.com/VUnit/vunit/blob/master/vunit/ui/__init__.py#L928 and https://github.com/VUnit/vunit/blob/master/vunit/verilog.py.That was required because of backwards compatibility. Although it is possible to disable
add_builtins
through optioncompile_builtins
, the default is to compile them (that is, data_types, core, logging, string_ops, check, dictionary, run and path). Therefore, when verilog support was added to VUnit, the existing function was preserved. Moreover,compile_builtins
was initially meant as a development feature (for running internal unit tests), and not to be explicitly used by consumers/users of VUnit.As discussed in #559, it would be better to expose the two explicit
add_*_builtins
in the UI, which would allow to get rid of thevunit.verilog
class. That is being addressed in #757. It does modify the public API (UI), butbuiltins.py
is not modified.array_util is deprecated since 2y ago, and it will be removed soon. See #600. The
_add_array_util
function inbuiltins.py
will be preserved, but it will raise an explicit RuntimeError. VHDL sources will be removed.Other deprecated VHDL sources will be removed (see #763), but those have no impact on
builtins.py
or the public API.In this context, the main disruptive change is that #764 will require all current users to use
add_vhdl_builtins
oradd_verilog_builtins
explicitly. As a result, most users will have snippets such as the following:On the one hand, I believe we can do better with regard to verbosity. I'd like to propose two alternatives which are not exclusive:
NOTE: We might want to discuss whether
add
is expressive enough.That is, allow specifying which optional libraries to compile through a list of strings, instead of requiring an explicit function for each of them. Passing a list of strings makes it easier to add/remove libraries in the future, as well as having aliases for them. For instance, we might support
vhdl
as an alias ofvhdl_builtins
andvcs
as an alias ofverification_components
. I would also agree with using an enumeration instead of strings, for better type checking.On the other hand, we might want to split
vhdl_builtins
intocore
andutils
. I don't mean removingvhdl_builtins
, but making it an alias ofutils
, which depends oncore
. The resulting list of keywords, packages and dependencies would be the following:data_types
core
dictionary
run
logging
string_ops
check
path
/cc @LarsAsplund @kraigher can you please confirm that this "tree" of dependencies is correct? I would like to add a diagram to the docs explaining this. My main doubts are:
run
package? Why is it not part ofcore
?logging
/string_ops
/check
/path
anddata_types
/core
/dictionary
/run
?verification_components
be explicitly dependent onrandom
?We can also take this opportunity to rethink the integration of third-party sources. We are currently using the core of OSVVM and JSON-for-VHDL through git submodules. Those are required because some of the features in VUnit utils or examples depend on them. However, users might want the same ease of addition for the whole OSVVM, for UVVM, SVUnit, or any other public or in-house library. The fact that users can already extend their setup through
add_source_files
seems to be overseen by multiple users and maintainers of other projects, and there is the unfortunate perception about some sources being first classs citizens.That was discussed in #511, and it was considered not convenient to embed git/wget/curl into the Python codebase of VUnit. I agree it is desirable to let users download the sources however they want, but I don't think each user should guess or copy and paste the snippet of
add_source_files
statements.Therefore, I believe that the approach should be documentation. As far as I am aware, there are two solutions for users to write reusable Python snippets that extend VUnit:
add
method proposed above, or add newadd_*
methods.from_argv
andfrom_args
methods to process some items from theadd
argument, before calling the methods of the parent (super?) class./cc @kraigher @Paebbels @ktbarrett, any thought?
As illustrated in OSVB: Overview, currently each verification framework/methodology uses a very different format for defining the sources:
.pro
files which are TCL code and need to be executed in a given order. Sources are listed in compilation order..txt
files. Sources are listed in compilation order.My initial intent some months ago was to use FuseSoC's
.core
files for replacing OSVVM's and UVVM's.pro
and.txt
sources. As a matter of fact, @Paebbels' vendor scripts (https://github.com/ghdl/ghdl/tree/master/scripts/windows) do parse those files already. The scripts are currently written in PowerShell and Bash, but we expect to convert them to Python (see ghdl/ghdl#1821). Hence, we have the knowledge for reading the files, but unfortunately don't have the knowledge for reading FuseSoC's.core
(CAPI 2) format. That's because there is no reusable and documented Python API for interacting with CAPI 2, there is no "language model". Therefore, I prototyped pyCAPI as a proof of concept for showcasing how might a FuseSoC-VUnit integration look like, should FuseSoC have a reusable Python API.After multiple discussions in the community, I/we decided not to waste time in a probably limited declarative format (either reused or invented), and instead use imperative scripts straightaway, i.e. Python scripts. That was one of the triggering factors for creating EDAA. pyEDAA.ProjectModel provides an abstraction that can be used for representing the whole OSVVM, UVVM and/or any other "third-party library". Therefore, writing a single ProjectModel-to-VUnitProject conversion function/class might suffice. OSVVM and UVVM would, on their own, provide Python scripts describing their sources as a ProjectModel, and VUnit would use a generic solution. I'm not sure about ProjectModel-to-VUnitProject being a good fit in the pyEDAA.ProjectModel repo. It should probably be contributed in this repository.
However, OSVVM and UVVM ProjectModel scripts were not written yet, neither the ProjectModel-to-VUnitProject conversion. Hence, it might be more straightforward to skip ProjectModel for now, and use the VUnit API directly. In the end, the main reason to use EDAA's API is because synthesis is out of scope of VUnit; otherwise, the EDAA ProjectModel would not be necessary. I.e., VUnit's solution is already an imperative script based approach using OOP.
My main doubt here is distribution. We don't want to add
vunit.osvvm
andvunit.uvvm
classes in this repo. If we were to do so, we might just add those to thebuiltins.py
, but that's the opposite of our purpose: we want a more generic solution, not double betting on manually supporting each third-party project. I'm thinking I might have those classes in OSVB temporarily (until we decide a better/definitive location). The problem is that OSVB is not published to PyPI, and I would honestly want to prevent users from considering OSVB a product per se. It is a place for discussion and to upstream enhancements, not a product. At the same time, I'm afraid that it can be problematic to force users to install some Python sources from OSVB in order to use the VUnit-OSVVM and VUnit-UVVM integrations.I think that the first "users" might be developers; the developers of OSVVM and UVVM. If they want to collaborate in this effort, I can provide a GitHub Action, a reference workflow and a
run.py
script which are ready-to-use, including something such as the following:The target would be to run three testbenches, at least, one using each framework/methodology.
That will allow all of us to better understand the challenges for reusing VUnit's Python codebase while best integrating with the features provided by OSVVM and/or UVVM. According to GitHub Facts About the HDL Industry: Wilson Study Comparison, it would get 60% of the FPGA market to use the same CI framework, regardless of the preferred testbench methodology and/or HDL utils.
/cc @Paebbels @JimLewis @mariuselv @UVVM
The text was updated successfully, but these errors were encountered: