Tweaks to the Builtins API

[umarcor]

In upcoming releases (#763), we expect to implement #559 (see #757 and #764), which will introduce some changes to the management of builtins. Builtins are the HDL libraries distributed with VUnit, some of which are required and some are optional. I believe we should take this opportunity to think about the API.

NOTE: all the information below is found in vunit/builtins.py.

Currently, the list of "modules/libraries" is the following:

• data_types
• core
• logging
• string_ops
• check
• dictionary
• run
• path
• array_util
• com
• verification_components
• random
• osvvm (submoduled)
• json4vhdl (submoduled)

The internal API provides a helper (named add), and the following explicit functions:

• _add_data_types
  • data_types
• _add_array_util
  • array_util
• _add_random
  • random
• _add_com
  • com
• _add_verification_components
  • verification_components
• _add_osvvm
  • osvvm
• _add_json4vhdl
  • json4vhdl
• add_verilog_builtins
  • vunit_pkg.sv
• add_vhdl_builtins
  • core
  • logging
  • string_ops
  • check
  • dictionary
  • run
  • path

Furthermore:

• _add_data_types is executed in _add_vhdl_builtins, and the following dependencies exist:
  • verification_components depends on com and osvvm.
  • random depends on osvvm.
• All the functions except the external code (osvvm and json4vhdl) use a helper (_add_files), because a file naming convention is used to indicate which standard is supported. That is something we cannot enforce in external projects.
• Function osvvm_is_installed requires OSVVM to be installed (the submodule to be checked out) so that VUnit can be used. That is more of an historical remainder than an a sensible solution nowadays.

Kind of suprisingly, the public API of VUnit (UI) exposes most of the functions listed above, except _add_data_types, add_verilog_builtins and add_vhdl_builtins. Instead, there is a single add_builtins that defaults to add_vhdl_builtins and it is overriden in class vunit.verilog. See https://github.com/VUnit/vunit/blob/master/vunit/ui/__init__.py#L928 and https://github.com/VUnit/vunit/blob/master/vunit/verilog.py.

That was required because of backwards compatibility. Although it is possible to disable add_builtins through option compile_builtins, the default is to compile them (that is, data_types, core, logging, string_ops, check, dictionary, run and path). Therefore, when verilog support was added to VUnit, the existing function was preserved. Moreover, compile_builtins was initially meant as a development feature (for running internal unit tests), and not to be explicitly used by consumers/users of VUnit.

As discussed in #559, it would be better to expose the two explicit add_*_builtins in the UI, which would allow to get rid of the vunit.verilog class. That is being addressed in #757. It does modify the public API (UI), but builtins.py is not modified.

array_util is deprecated since 2y ago, and it will be removed soon. See #600. The _add_array_util function in builtins.py will be preserved, but it will raise an explicit RuntimeError. VHDL sources will be removed.

Other deprecated VHDL sources will be removed (see #763), but those have no impact on builtins.py or the public API.

---

In this context, the main disruptive change is that #764 will require all current users to use add_vhdl_builtins or add_verilog_builtins explicitly. As a result, most users will have snippets such as the following:

# example array_axis_vcs
VU = VUnit.from_argv()
VU.add_vhdl_builtins()
VU.add_verification_components()

# example axi_dma
VU = VUnit.from_argv()
VU.add_vhdl_builtins()
VU.add_osvvm()
VU.add_verification_components()

# example com
VU = VUnit.from_argv()
VU.add_vhdl_builtins()
VU.add_com()
VU.add_verification_components()
VU.add_osvvm()

On the one hand, I believe we can do better with regard to verbosity. I'd like to propose two alternatives which are not exclusive:

# example axi_dma
VU = VUnit.from_argv()
VU.add('vhdl_builtins', 'osvvm', 'verification_components')

# example axi_dma
VU = VUnit.from_argv(add=['vhdl_builtins', 'osvvm', 'verification_components'])

NOTE: We might want to discuss whether add is expressive enough.

That is, allow specifying which optional libraries to compile through a list of strings, instead of requiring an explicit function for each of them. Passing a list of strings makes it easier to add/remove libraries in the future, as well as having aliases for them. For instance, we might support vhdl as an alias of vhdl_builtins and vcs as an alias of verification_components. I would also agree with using an enumeration instead of strings, for better type checking.

On the other hand, we might want to split vhdl_builtins into core and utils. I don't mean removing vhdl_builtins, but making it an alias of utils, which depends on core. The resulting list of keywords, packages and dependencies would be the following:

• core:
  • data_types
  • core
  • dictionary
  • run
• utils ([core]):
  • logging
  • string_ops
  • check
  • path
• com ([utils])
• random ([utils, osvvm])
• verification_components ([utils, com, random, osvvm])
• osvvm
• json4vhdl

/cc @LarsAsplund @kraigher can you please confirm that this "tree" of dependencies is correct? I would like to add a diagram to the docs explaining this. My main doubts are:

• Is it possible to use VUnit without the run package? Why is it not part of core?
• Do you have any clear guess about the dependencies between logging/string_ops/check/path and data_types/core/dictionary/run?
• Should verification_components be explicitly dependent on random?

---

We can also take this opportunity to rethink the integration of third-party sources. We are currently using the core of OSVVM and JSON-for-VHDL through git submodules. Those are required because some of the features in VUnit utils or examples depend on them. However, users might want the same ease of addition for the whole OSVVM, for UVVM, SVUnit, or any other public or in-house library. The fact that users can already extend their setup through add_source_files seems to be overseen by multiple users and maintainers of other projects, and there is the unfortunate perception about some sources being first classs citizens.

That was discussed in #511, and it was considered not convenient to embed git/wget/curl into the Python codebase of VUnit. I agree it is desirable to let users download the sources however they want, but I don't think each user should guess or copy and paste the snippet of add_source_files statements.
Therefore, I believe that the approach should be documentation. As far as I am aware, there are two solutions for users to write reusable Python snippets that extend VUnit:

• A function that receives and returns a VUnit object. That is used in umarcor/osvb. Precisely, an AXI4Stream.core file is used to define the sources declaratively, and a helper function adds the content to the VU object. See https://github.com/umarcor/osvb/blob/main/AXI4Stream/test/vunit/run_capi.py#L15-L19 and https://github.com/umarcor/osvb/blob/main/mods/pyCAPI/VUnit.py.
  • This is probably the most naive approach. It is functional, easy to understand and to use by users without strong software knowledge.
• A class extending the VUnit UI. The soon to be deprecated vunit.verilog class showcases how to override a method of the public API. There are three possible non-exclusive approaches:
  • Override the add method proposed above, or add new add_* methods.
  • Override the from_argv and from_args methods to process some items from the add argument, before calling the methods of the parent (super?) class.
  • Override the Builtins class/object. I know this needs to be technically possible, but I'm unsure about the complexity or whether there is an actual use case for it.

/cc @kraigher @Paebbels @ktbarrett, any thought?

---

As illustrated in OSVB: Overview, currently each verification framework/methodology uses a very different format for defining the sources:

• VUnit uses the functions/API explained above.
• OSVVM uses .pro files which are TCL code and need to be executed in a given order. Sources are listed in compilation order.
• UVVM uses .txt files. Sources are listed in compilation order.
• Cocotb uses makefiles.

My initial intent some months ago was to use FuseSoC's .core files for replacing OSVVM's and UVVM's .pro and .txt sources. As a matter of fact, @Paebbels' vendor scripts (https://github.com/ghdl/ghdl/tree/master/scripts/windows) do parse those files already. The scripts are currently written in PowerShell and Bash, but we expect to convert them to Python (see ghdl/ghdl#1821). Hence, we have the knowledge for reading the files, but unfortunately don't have the knowledge for reading FuseSoC's .core (CAPI 2) format. That's because there is no reusable and documented Python API for interacting with CAPI 2, there is no "language model". Therefore, I prototyped pyCAPI as a proof of concept for showcasing how might a FuseSoC-VUnit integration look like, should FuseSoC have a reusable Python API.

After multiple discussions in the community, I/we decided not to waste time in a probably limited declarative format (either reused or invented), and instead use imperative scripts straightaway, i.e. Python scripts. That was one of the triggering factors for creating EDAA. pyEDAA.ProjectModel provides an abstraction that can be used for representing the whole OSVVM, UVVM and/or any other "third-party library". Therefore, writing a single ProjectModel-to-VUnitProject conversion function/class might suffice. OSVVM and UVVM would, on their own, provide Python scripts describing their sources as a ProjectModel, and VUnit would use a generic solution. I'm not sure about ProjectModel-to-VUnitProject being a good fit in the pyEDAA.ProjectModel repo. It should probably be contributed in this repository.

However, OSVVM and UVVM ProjectModel scripts were not written yet, neither the ProjectModel-to-VUnitProject conversion. Hence, it might be more straightforward to skip ProjectModel for now, and use the VUnit API directly. In the end, the main reason to use EDAA's API is because synthesis is out of scope of VUnit; otherwise, the EDAA ProjectModel would not be necessary. I.e., VUnit's solution is already an imperative script based approach using OOP.

My main doubt here is distribution. We don't want to add vunit.osvvm and vunit.uvvm classes in this repo. If we were to do so, we might just add those to the builtins.py, but that's the opposite of our purpose: we want a more generic solution, not double betting on manually supporting each third-party project. I'm thinking I might have those classes in OSVB temporarily (until we decide a better/definitive location). The problem is that OSVB is not published to PyPI, and I would honestly want to prevent users from considering OSVB a product per se. It is a place for discussion and to upstream enhancements, not a product. At the same time, I'm afraid that it can be problematic to force users to install some Python sources from OSVB in order to use the VUnit-OSVVM and VUnit-UVVM integrations.

I think that the first "users" might be developers; the developers of OSVVM and UVVM. If they want to collaborate in this effort, I can provide a GitHub Action, a reference workflow and a run.py script which are ready-to-use, including something such as the following:

VU = VUnit.from_argv()
VU.add([
    # VUnit minimal
    'vhdl_builtins',
    # VUnit extended
    'verification_components',
    # OSVVM minimal
    'osvvm',
    # OSVVM extended
    'osvvm_libraries',
    # UVVM minimal
    'uvvm_light',
    # UVVM extended
    'uvvm',
])

The target would be to run three testbenches, at least, one using each framework/methodology.
That will allow all of us to better understand the challenges for reusing VUnit's Python codebase while best integrating with the features provided by OSVVM and/or UVVM. According to GitHub Facts About the HDL Industry: Wilson Study Comparison, it would get 60% of the FPGA market to use the same CI framework, regardless of the preferred testbench methodology and/or HDL utils.

/cc @Paebbels @JimLewis @mariuselv @UVVM