Dynamic discovery of technology-dependent primitives #1679
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
imphil
requested review from
eunchan,
msfschaffner,
sjgitty and
tjaychen
as code owners
|
@msfschaffner can you try this PR and see if you're able to produce a technology library for your preferred ASIC target? |
|
Thanks for the PR, @imphil! Will give this a go tomorrow. |
I won't get around to it today most likely; so feel free to have an initial look, but don't expect it to fully work yet 😄 |
imphil
force-pushed
the
primgen
branch
4 times, most recently
from
d0942dc
to
a871a32
Compare
|
OK. Should be all sorted now, over to you @msfschaffner for testing! |
imphil
force-pushed
the
primgen
branch
2 times, most recently
from
72e9b57
to
7f3ccb8
Compare
imphil
force-pushed
the
primgen
branch
2 times, most recently
from
e4feccd
to
d7fbd2a
Compare
|
Looks like Private CI is useful! Fixed a case where I didn't update the path of a generated flash memory in the chip DV environment. |
|
Reran with the newest version. It seems to work fine for our usecase (also the degenerate The only thing I keep on seeing are the following warnings: the warnings persist even if I set the filetype to |
|
OK, the sporadic failures in private CI are caused by a race condition in fusesoc when creating the generated core files/primitives in multiple fusesoc runs at the same time. I'll look into it. |
4 tasks
|
Welcome to the rabbit hole ...
The core dependency issue needs to be merged first before CI runs on this one can pass. |
|
Ok, SGTM! |
Some primitives are intended to be implemented in a technology-specific way, e.g. differently for Xilinx or Altera FPGAs. Which implementation is to be used can be specified at synthesis time. Target-specific implementations are collected in "technology libraries", or "techlibs". When writing code, only "abstract primitives" are used. These are then implemented by a techlib. Currently, all techlibs are part of the OpenTitan repository, and all abstract primitives are hand-crafted. This commit adds a script to auto-generate these abstract primitives. The script is to be used as FuseSoC generator. Note that the script currently is in a rather "rough" form. The SystemVerilog "parsing" is done using a regex instead of a proper parser, and speed isn't great either. Signed-off-by: Philipp Wagner <phw@lowrisc.org>
Instead of handcoding the abstract primitives, they are now auto-generated through the primgen script, hooked up as fusesoc generator. This requires a not-yet-released version of fusesoc to have all information available that our generator needs. Use `pip3 install --user -U -r python-requirements.txt` to get a supported fusesoc version for this change. After this commit, technology libraries can simply be added to the fusesoc search path. A technology library is expected to be similar to the `prim_xilinx` library we have in the tree today; copy this library as a starting point. ``` fusesoc --cores-root=. --cores-root=path-to-your-other-techlib run .... ``` Signed-off-by: Philipp Wagner <phw@lowrisc.org>
This is a starting point for documenting the primmitives system, especially regarding technology-dependent primitives and how they are used/generated. Signed-off-by: Philipp Wagner <phw@lowrisc.org>
|
So, this should be the last update; I'm reasonably convinced that the implementation was/is correct and #1750 was really just a side effect of a previous implementation detail. To make our life easier tracking down dependency issues in the future I've included a patch to fusesoc which always dumps a
|
|
Oh wow, looks like you had a hell of a debugging evening! |
imphil
added a commit
to lowRISC/fusesoc
that referenced
this pull request
Apr 6, 2020
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
msfschaffner
pushed a commit
to msfschaffner/fusesoc
that referenced
this pull request
Apr 17, 2020
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
rswarbrick
pushed a commit
to rswarbrick/fusesoc
that referenced
this pull request
Jun 12, 2020
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
rswarbrick
pushed a commit
to rswarbrick/fusesoc
that referenced
this pull request
Jul 1, 2020
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
rswarbrick
pushed a commit
to lowRISC/fusesoc
that referenced
this pull request
Sep 23, 2020
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
imphil
added a commit
to lowRISC/fusesoc
that referenced
this pull request
Mar 24, 2021
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
imphil
added a commit
to lowRISC/fusesoc
that referenced
this pull request
Oct 1, 2021
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
vogelpi
pushed a commit
to vogelpi/fusesoc
that referenced
this pull request
Jul 15, 2022
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
bwitherspoon
pushed a commit
to bwitherspoon/edalize
that referenced
this pull request
Dec 17, 2023
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
bwitherspoon
pushed a commit
to bwitherspoon/edalize
that referenced
this pull request
Dec 17, 2023
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
bwitherspoon
pushed a commit
to bwitherspoon/fusesoc
that referenced
this pull request
Dec 17, 2023
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
bwitherspoon
pushed a commit
to bwitherspoon/fusesoc
that referenced
this pull request
Mar 9, 2024
Generators now can be used to dynamically generate other cores depending on which cores were discovered during a fusesoc run. To enable this functionanlity, a list of all discovered cores is passed to the generator inside the GAPI file structure. This modifies the GAPI schema; but since it is a strict addition of a key which shouldn't affect users who did not use this functionality before, the GAPI version is not incremented. The motivating need for this work can be found in lowRISC/opentitan#1679. In short: RTL primitives, like a RAM module, exist in various specializations for different target technologies (e.g. one RAM implementation for Xilinx FPGAs, another one for Altera FPGAs). Such target-specific implementations of the RAM primitive are collected in a technology library. Not all users have access to all technology libraries, that's why these libraries need to be plug-and-play. A generator is now used to create a "wrapper primitive". The wrapper primitive is a simple RTL file with if/else blocks, choosing a target-specific implementation based on a parameter. To create this wrapper primitive, fusesoc searches through its list of core library paths, and passes all cores it found to a generator. The generator picks out (e.g. based on a standardized naming scheme) the implementations of a given primitive, and includes them in the "wrapper primitive".
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, all primitives are implemented in-tree. This is a reasonable approach as long as we can share them. But that's not the case for many ASIC targets. We therefore need a way to keep the target-dependent implementations of primitives out of the tree. At the same time, we don't want to hard-code dependencies on these targets, as different users of OpenTitan might have different libraries available.
This pull request solves the above problem by introducing the dynamic discovery of "technology libraries", groups of implementations of our primitives. The implementation heavily relies on FuseSoC generators, which enables us to dynamically discover technology libraries by simply making another directory of cores (a "FuseSoC library") known to FuseSoC. This can be done by adding another
--cores-rootparameter to a fusesoc call, or by setting theFUSESOC_CORESenvironment variable (multiple directories can be specified colon-separated, like inPATH).Note that this implementation relies on features in FuseSoC which are not yet upstreamed. It is therefore necessary to update fusesoc by running
Further documentation for this feature is contained in the
hw/ip/prim/README.mdadded as part of this pull request. Please also have a look at the individual commit messages.Note: This PR currently contains #1671. It will be rebased to remove these commits once that PR lands.