Pass a list of all discovered cores to a generator

Generators now can be used to dynamically generate other cores depending
on which cores were discovered during a fusesoc run. To enable this
functionanlity, a list of all discovered cores is passed to the
generator inside the GAPI file structure.

This modifies the GAPI schema; but since it is a strict addition of a
key which shouldn't affect users who did not use this functionality
before, the GAPI version is not incremented.

The motivating need for this work can be found in
lowRISC/opentitan#1679. In short: RTL
primitives, like a RAM module, exist in various specializations for
different target technologies (e.g. one RAM implementation for Xilinx
FPGAs, another one for Altera FPGAs). Such target-specific
implementations of the RAM primitive are collected in a technology
library. Not all users have access to all technology libraries, that's
why these libraries need to be plug-and-play.

A generator is now used to create a "wrapper primitive". The wrapper
primitive is a simple RTL file with if/else blocks, choosing a
target-specific implementation based on a parameter. To create this
wrapper primitive, fusesoc searches through its list of core library
paths, and passes all cores it found to a generator. The generator
picks out (e.g. based on a standardized naming scheme) the
implementations of a given primitive, and includes them in the "wrapper
primitive".

doc/source/user/generators.rst

@@ -44,6 +44,12 @@ Example yaml configuration file:
         spi0: {datawidth: 8, offset: 2952790016, size: 8}
         uart0: {datawidth: 8, offset: 2415919104, size: 32}
     vlnv: ::mysoc-wb_intercon:0
+    cores:
+      ::SD-card-controller:0:
+        capi_version: 1
+        core_filepath: path/to/a/file.core
+        used: false
+
 
 
 The above example is for a generator that creates verilog code for a wishbone interconnect.

fusesoc/edalizer.py

@@ -128,6 +128,36 @@ def extract_generators(self):
 
         self.generators = generators
 
+    def _invalidate_cached_core_list_for_generator(self):
+        if self._cached_core_list_for_generator:
+            self._cached_core_list_for_generator = None
+
+    def _core_list_for_generator(self):
+        """Produce a dictionary of cores, suitable for passing to a generator
+
+        The results of this functions are cached for a significant overall
+        speedup. Users need to call _invalidate_cached_core_list_for_generator()
+        whenever the CoreDB is modified.
+        """
+
+        if self._cached_core_list_for_generator:
+            return self._cached_core_list_for_generator
+
+        out = {}
+        resolved_cores = self.resolved_cores  # cache for speed
+        for core in self.discovered_cores:
+            core_flags = self._core_flags(core)
+            out[str(core)] = {
+                "capi_version": core.capi_version,
+                "core_filepath": os.path.abspath(core.core_file),
+                "used": core in resolved_cores,
+                "core_root": os.path.abspath(core.core_root),
+                "files": [str(f["name"]) for f in core.get_files(core_flags)],
+            }
+
+        self._cached_core_list_for_generator = out
+        return out
+
     def run_generators(self):
         """ Run all generators """
         generated_libraries = []
@@ -137,7 +167,12 @@ def run_generators(self):
 
             if hasattr(core, "get_ttptttg"):
                 for ttptttg_data in core.get_ttptttg(core_flags):
-                    ttptttg = Ttptttg(ttptttg_data, core, self.generators,)
+                    ttptttg = Ttptttg(
+                        ttptttg_data,
+                        core,
+                        self.generators,
+                        core_list=self._core_list_for_generator(),
+                    )
                     gen_lib = ttptttg.generate(self.cache_root)
 
                     # The output directory of the generator can contain core
@@ -163,6 +198,7 @@ def run_generators(self):
         # cache and is therefore quite expensive.
         for lib in generated_libraries:
             self.core_manager.add_library(lib)
+        self._invalidate_cached_core_list_for_generator()
 
     def create_eda_api_struct(self):
         first_snippets = []
@@ -377,7 +413,7 @@ def to_yaml(self, edalize_file):
 
 
 class Ttptttg:
-    def __init__(self, ttptttg, core, generators):
+    def __init__(self, ttptttg, core, generators, core_list):
         generator_name = ttptttg["generator"]
         if not generator_name in generators:
             raise RuntimeError(
@@ -405,6 +441,7 @@ def __init__(self, ttptttg, core, generators):
             "gapi": "1.0",
             "parameters": parameters,
             "vlnv": vlnv_str,
+            "cores": core_list,
         }
 
     def generate(self, cache_root):