Add Configurable Generators

[benreynwar]

Configurable Generators

At the moment all the parameters for a 'generator' must be fully
specified in the core file with the corresponding 'generate' (ttctg).

It would be useful to have a 'generate' that doesn't specify the
parameters of the generator. We'll call generators that don't require
the 'generate' to specify the parameters as 'Configurable Generators',
as opposed to the current 'Nonconfigurable Generators'.

Configurable Generators can be configured in four ways;

• From the 'generate' like a nonconfigurable generator.
• From the command line if it is the top-level core.
• From a parent generator.
• From generics/parameters specificed in a V*HDL file in the parent fileset.

While making this change it also makes sense to add the ability for
cores to pass parameters to their parent generators (see issue #250)
Both configurable and unconfigurable generators would receive these
parameters.

The motivation for these changes are described in issue #248 which I'm
replacing with this issue.

Configuration Implementation

Each core contains filesets and generates(ttctg). These
filesets/generates within each core can be arranged into a dependency
order. Each fileset or generate can have dependencies on other cores.

1. Build a dependency tree of filesets/generates. A dependency on a
   core, is changed to a dependency on the fileset/generate at the
   top of that cores dependency order.

2. Generators are divided into configurable and nonconfigurable
   generators. We classify filesets and configurable generators as
   nonconfigurable components. Configurable generators are
   configurable components.

3. We now have a dependency tree containing configurable and
   nonconfigurable components. Nonconfigurable components that are
   directly connected in the dependency tree are grouped into
   neighbourhoods. We now have a dependency tree of configurable
   generators and neighbourhoods.

4. Each configurable generator and neighbourhood has a 'configure'
   method. A configure method receives a set of configuration
   parameters from it's parent. During execution a configure method
   calls the configure methods of the children. These configure calls
   return parameters. It saves it's state, and returns parameters to
   it's parent. How this is done is different for a configurable
   generator vs a neighbourhood. For a configurable generator the
   'configure' method just calls the 'configure_command' specified
   in it's core file.

   Fileset neighbourhoods receive parameters either from the command
   line (if they are at the top-level) or from a parent generator. If
   a fileset neighbourhood has no child generators, then all the
   configure method does is directly return any 'back_parameters'
   defined in the top core. If a fileset neighbourhood has any child
   generators that need to receive parameters then a simulation of the
   neighbourhood is run using 'target=configure'. The child generators
   must have the 'configurable_from_fileset' property. This means that
   when 'target=configure' they generate a stub file that logs the
   generics/parameters with which they were instantiated to stdout.
   The generics/parameters are extracted from stdout and are then
   passed to the appropriate generator configure calls as parameters.
   If the fileset neighbourhood contains nonconfigurable generators
   then these must be run before the simulation is run.

5. Because generators can save state on a call to configure a second
   call to configure can produce different parameters. Using this it
   is possible to support the chisel diplomacy protocol. The
   convention is that if the top-level generator returns a parmeter of
   'run_config_again'=True then configure should be run again.

Generation

Once configuration is complete generation is run. Any unconfigurable
generators that haven't been run yet are run. The generate methods
on the configurable generators are called.

Changes Required for CAPI2 core definition

Although this involves a bunch of changes to fusesoc itself it
has a pretty minimal effect on the core definitions.

fileset

• add 'back_parameters'

generator

• add 'configure_interpreter'
• add 'configure_command' (the presense of this makes the generator configurable).
• add 'configurable_from_fileset'
• add 'depend'

generate

• add 'back_parameters'

[olofk]

Hi @benreynwar I had totally missed this one. Just writing to know that I've seen it now. Unfortunately I have only taken the time for a quick look and not yet had time to think properly about your suggestions

[benreynwar]

@olofk This is two years old so I don't really remember what I was thinking. It all looks a bit complicated looking at it now, but perhaps there are some useful ideas in there :).

[olofk]

Haha. Before seeing your latest comment, I just started reading through it thinking it all looked very complicated and that it would need to take time to wrap my head around it :)

As you say, it might be too complicated, I'm not sure at this point. One point that directly stood out is how we can intertwine verilog/vhdl parameters/generics with generator parameterization (is that a word?). I think that this could be reasonably solved with variable substitution in the core files which is on the roadmap. The big thing I think we haven't solved is the diplomacy aspect of passing configuration upstream. That's one tricky bit and I honestly hope we can get around it in other ways instead

[benreynwar]

Yep. I agree that generics to generator parameterization would be really useful and that the upstream configuration is more trouble than it's worth.

I mainly need generics to generator parameterization for configuring compiled memories. Currently I do this creating a stub vhdl file for my memories that just logs the depth and width of the memory to stdout. I call fusesoc to generate all my files, then use ghdl to run a 0 clock cycle simulation which causes all the widths and depths of my memories to get logged to stdout. I have a little script that then collects these widths and depths and writes them to a file. I then call fusesoc again and this time the memory generator uses that file to generate all the appropriate memories. It works, but it's a bit ugly. I'm unsure what the best way to integrate this kind of functionality into fusesoc is, or how generally useful it would be.

On a slightly higher level, we have all these cool hdl generators like migen and chisel, but it's not clear how a chisel design would instantiate a migen design, if the parameters for the migen design are some function of the parameters for the chisel design. Fusesoc feels like a good place to create conventions for how to pass elaboration parameters through a hierarchy of generators. Realistically I don't think this is actually a real problem yet :). Hopefully in the future we'll have lots of open-source designs that depend on each other and we'll have to start worrying about it!

Feel free to close this issue, since I don't think there is any direction action that needs to be taken for it.

[cr1901]

I've run into this with my migen_uart core:

• I have a design, which uses an FPGA-internal clock. I.e. clock speed is configurable at the command line.
• I want to add a parameter to change the clock speed. Why? Because the ufm_reader will need wait states at higher clocks, and I wanted to use fusesoc to quickly test this :).
• The migen_uart core requires fusesoc to tell it the clock speed it runs at. Right now, there's no way to pass this information from the command line short of remembering to manually update the generate parameters in ufm_reader.core every time I want to test a different clock speed. I already know I'm going to screw this up and forget.