-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Cider 2] Handle memory input/output #1968
Comments
I am currently leaning toward having a simple preamble with this info since I think it will make things simpler in the long run, but I am open to other ideas. CC: @sampsyo |
All sounds awesome. I agree with the decisions you summarized briefly:
And very good reasoning about the metadata that describes what's in the file(s), which of course is necessary for producing any other format from such a binary dump. I actually think this dovetails nicely with two other things going on in the ecosystem. First:
This is pretty much the goal of the "YXI" interface definition format created by @nathanielnrn for AXI purposes! Check it out:
That is, YXI is just a JSON document that includes the names and dimensions of the exposed top-level memories (using So, Cider 2.0 could dump "just the bytes" and rely on a separate YXI file to interpret it. Or, it could produce a single file that consists of the YXI data (presumably serialized to some other format) followed by all the bytes. Either way, maybe it would be cool to standardize on YXI being the way to describe this stuff, extending it if necessary to address this use case (as opposed to inventing a new/different format with similar-but-not-quite-identical contents)?
I don't think I've broadcasted this too broadly, but @bcarlet and I have recently started working with @Angelica-Schell to do something like this!! That is, we are starting small, but we are hoping to build a standalone data converter tool (as you say, taking the fud2 approach) that can convert between many different data formats. Including Verilog-simulator-friendly binary files, OG fud-style JSON, anything else we can think of. And Cider's preferred format could be wrapped up into that! Anyway, this is just to say that we should totally build such a thing, and it should probably be a command-line flag tacked onto what @Angelica-Schell is beginning to construct now. |
Ah brilliant, that's exactly what I was thinking about. Happy to help out if needed.
Amazing! How does this look for multidimensional memories with the size definition? Essentially what I was imagining was basically the yxi interface info (serialized into a binary format) followed by the raw binary data for all the memories one after another. I think that is better than the version where we have the header for a single memory followed by the data for that memory and so on, since we can easily extract the header info from that without needing to parse the entire file. |
I don't remember of the top of my head where we landed on multi-dimensional memories, but IIRC we either (a) don't handle them at all, or (b) just report the total size (i.e., the product of the dimensions). And glancing quickly at the code, I think it's (b), as in, it uses Line 39 in 23cd0a1
Yeah, totally makes sense to me. |
Currently Cider 2.0 cannot print out the contents of memory in a way that is compatible with the json tooling we use for snapshot testing. The old infrastructure for this is somewhat tangled and tortured in both directions since it involves some base64 encodings and a nightmare fud python script. Rather than hooking back into this, it could be worthwhile to do something different for the new version, specifically a more "stupid" binary encoding.
@sampsyo said:
There are a few things to pin down for this since we can have memories contain values of arbitrary bit width.
To that end I propose the following:
The non-obvious part is what we do about structure and validation. We could make the dump be as dumb as possible, i.e., just bits with no embedded information about memory names or structures, but that requires us to have a data file available to deserialize the dump and means for loading memory we just have to assume that the data file was created with the memory information laid out correctly and in the exact order in which the memory instances are defined in the main component. To that end, it may be worth having a preamble which contains the names of the memories, their dimensions, and the definition order and have a tool binary, in the fud2 philosophy, which can convert json to memory dumps and vice-versa without requiring retaining a data file (or constructing a dummy one for the cases in which we run a program without providing data and still wish to observe the output)
The text was updated successfully, but these errors were encountered: