automate inclusion of BBF operators in verilator

[stevobailey]

i don't want to copy the verilog everytime i run verilator in some higher-level project. how do i do this?

[grebe]

In particular, even if a dsptools driver spits out verilog in the right place, it won't work with the rocket build system.

@chick, do you have any thoughts? I'm wondering if a firrtl pass that inlines the handwritten verilog would be a workable solution.

[chick]

@stevobailey @grebe : I don't know enough to comment on this. I'd be happy to help work on it though.

[grebe]

this code seems to be the only place that refers to the verilog black box implementations for floating point and I'm not even really sure what it's doing. There should be some standard way of getting the verilog blackboxes where they need to be b/c at the moment we're copying the file manually as needed, which is silly.

The rocket build system isn't particularly relevant, other than to say that having something in Driver write out BlackBoxFloat.v won't really fix it b/c rocket invokes its own driver.

Actually, if you run a test with floating point using the verilator backend, how does it get the blackbox implementations right now?

[stevobailey]

Verilator looks in the current directory and the test_run_dir/**/ directory for files named "Module.v" where Module is the missing module, such as BBFMultiply or BBFAdd. To get it to work, I manually copy the file into the test_run_dir and rename it. We could have firrtl or whatever creates the Verilog file automatically write out these files along with the design into the appropriate test_run_dir. Or we can inline the modules as Paul suggested earlier. How does the design Verilog get in the test_run_dir?

(This sort of hints at a bigger change others have suggested, namely separating modules into individual Verilog files called "Module.v". But I think that's a separate discussion.)

[shunshou]

I believe the particular directory is set via execution options somewhere, but exactly where [and what's the default] is complicated to dig through, hence why I really want someone to document how things work.

I think I'll need something like this for other reasons, and I'm down to write a FIRRTL pass that inlines black boxes, if you want... since I really don't want to start porting my stuff until all of my concerns re: ranging, execution options documentation, and rounding are actually addressed. I basically can't do anything interesting that I need without rounding, and it's just annoying the heck out of me to figure out how to set up custom driver options when, really, someone who knows it, can just spend an hour documenting.

Also, I really think things in DSPtools should be separated out. It's kind of a mess and poorly unorganized, and it'll only get worse if you keep dumping new features in b/c of tight deadlines or w/e.

[ducky64]

So Chisel has options in the testrunner to include external .v files, which is how black box tests are done in core Chisel. I don't know enough about rocket-chip, but would it be feasible to have rocket-chip tests be more like Chisel tests, using a core-Chisel derived testrunner (rather than doing its own thing way off yonder)?

[shunshou]

This whole Chisel/rocket-chip thing is just super, as Paul would put it, janky. The infrastructure to test with rocket-chip at the top level is completely different from running tests on an isolated module with Chisel... but I think it's too legacy-ish (and possibly necessary to keep?)... At least that was my observation from dealing with rocket-chip in the summer. But it's not just tests; it's the whole build structure... I assume rocket-chip still passes in params [like the ReplSeqmem] via command line arguments... so rocket-chip is just not a good example going to the future, IMO.

On a complete side note/hijacking the conversation, @ducky64 do you have a cute mini example of multiclock with Chisel3? [I guess the hack before they add in formalized API...] and specifically how to test with Verilator? I have a lot of divided down clock domains that I need in my design, so it'd be nice if I had an example starting point... And is there any status update on the API?

Anyways, swapping out blackboxes with a simple FIRRTL pass should be pretty easy. Swapping out general modules is also useful, and I don't think doing something like that would be considered a "hack" solution.

[ducky64]

So the engineer in me would argue for improving the rocket-chip infrastructure, because everyone and their dog is going to want a open processor in their chip, and have it well tested in a coherent framework.
The researcher in me rates the probability of this happening as pretty low, unless we had a coordinated infrastructure push or there was a good research justification (such as testability, if our advanced testing tools require chisel3's new infrastructure). But it may be worth seeing if people are interested in a coordinated infrastructure push; I imagine everyone else using rocket-chip is also just as annoyed with the lack of standardization.

Derailing this thread for clock-crossing: my JTAG stuff does that. Here's an example top-level design with module boundary (through override_clock and override_reset) clock-crossing with clock-crossing queues (AsyncDecoupledFrom) https://github.com/ucb-art/builtin-debugger/blob/master/examples/ice40hx8k-yosys/src/main/scala/main.scala
A minimal set of clock-crossing primitives is here, extracted from rocket-chip's infrastructure: https://github.com/ucb-art/chisel-jtag/tree/master/external
No updates on the API. It's supposed to happen, and I really hope it happens soon, but that's all I know...

[shunshou]

I don't disagree, but good luck getting some "coordinated infrastructure push" going. I'm just jaded :P

Basically, my own progress with anything Chisel-related has been generally hindered by "[hoping] it happens soon" == crickets chirping :D Either that or do it yourself with some hack solution that eventually gets replaced. Lots of redundant work... I get that agile design treats code as cheap... but rewriting the same code 4+ times just to fit with a standard at a given time doesn't seem to really be right either...

I'm more curious about how you test multi-clock. I recall you said you had to use Verilator, but was that with the peek/poke interface or did you have to write a separate Verilog TB? I like the peek/poke tester, and it'd be a pain to have to separately write VerilogTB's or ChiselTB's depending on which submodule I'm unit testing.

[ducky64]

One way to work around "hope it happens soon" is to open an issue with a proposal, bug people about it (either by including them on the issue, or finding them at the Chisel meetings), then implement and PR it. The proposal gives people a chance to give feedback, hopefully ensuring that there's no API changes on the horizon that would completely break it and that it's overall a good design that will last. Of course, this does take more time than writing throwaway code, and it may or may not be favorable compared to rewriting code 4+ times.

as the train continues to slide off its tracks: testing clock-crossing can be done by poke-ing the clock signal directly, using step() to propagate logic, and using the Verilator backend. JTAG example here: https://github.com/ucb-art/chisel-jtag/blob/master/src/test/scala/JtagTapTest.scala

Side: FIRterpreter is supposed to support multiclock, but whether it's there yet is unclear. It was broken at least when I tried it, or maybe I used it incorrectly.

[shunshou]

Ok, I now remember staring at your code awhile back and seeing the clock poking. Thanks :)

I think your solution is very idealized. I haven't opened that many issues, but I've opened enough and commented enough in others to know that most of the more important issues don't usually get dealt with. Since they're usually on major missing components of Chisel, at least from the few Chisel meetings I've gone to, they usually get deferred b/c of "major API changes" required... (or I guess Chisel2/Chisel3 transition priorities)... Small bug fixes are another story; they're easy to PR. But in general, I feel like people must've tried something similar, got ignored, and then ended up forking things on their own.

I'm not sure if Chisel people realize how urgent something like multiclocking is. From what I know, it's been a constant pain point since whenever Raven/Hurricane first needed multiple clocks... but apparently wasn't important enough for it to ever be resolved in Chisel? Or was it just thought to be too hard? idk. [See my original issue @ https://github.com/ucb-bar/chisel2-deprecated/issues/667 I'm sure there were others...] Sure, people can hack a solution together, but that's wasted time every time that has to occur... Same with all of my ChiselDSP requests... Granted, that was more of an issue b/c of Chisel2/3 transitioning... Or the particular example that bit me the hardest: the fact that ReplSeqMem didn't exist for my own tapeout :D... Although that probably was due to poor communication on all fronts and the fact that lots of people were away on internships... but Ben did post an issue!

As for PR's, esp. with major API changes, as you said before, it's kind of hard for people to want to code review something that's several thousand lines long cough my old ChiselDSP cough, and even if they did, it's something domain-specific enough where it's not all that easy to do, so either it gets approved without much review or gets tossed into the code graveyard.

If, somehow, everyone's expectations/goals could magically align (or people were just overly nice about giving up their time to help others), then we'd have a much more useful, far more productive tool. If you can get that to happen, I'll buy you dinner. :P

[ducky64]

I think that there are a lot of issues and there aren't enough Chisel core devs to deal with everything. Multiclock, for example, is something that's been on the radar for years, and there's been some (very slight) external pressure for it, but overall there's never been a clear internal driver for it and most could work around it with a janky (but correct) hack. The ball has been dropped on the clean API several times, though in general I think it's just that it's low priority, we're bad at estimating times to complete a task, and we may be overly optimistic about assigning tasks.

I'd also argue that a way to make your interests represented is to participate in the process (i.e. Chisel meetings, coordinated dev on core tools). Of course, not all solutions have clean broad agreement (addition width growth behavior being an example of unresolvable conflict between CPU architects and DSP architects), but many problems do. And it's easier to force people to respond to an API change proposal on the spot during a meeting compared to just through a GitHub issue. The hope would be that ultimately the time invested in process pays off in code not rewritten and code easier written, but if nothing else, there's food...

For a DSP framework in particular, it may not make sense to upstream that to Chisel, but it does depends on Chisel and continued stability of some API features. The Chisel API is still in a state of flux, and some features may be removed (like addPin), but hopefully it results in a cleaner, safer API that supports the necessary features through another mechanism - but someone has to be there to advocate for their use case.

With rocket-chip in particular, it gets messier (because SiFive is an external entity), but it might be interesting to see if other people also consider the infrastructure style a continuing problem, and if anyone is interested in fixing it once and for all instead of putting up with badness over and over (and over) again. Because I certainly would like to be able to instantiate a processor and write tests from the comfort of ScalaTest and sbt rather than the abomination that is the make language.

[shunshou]

Well, I think part of the reason I'm slightly averse to going to Chisel meetings is... there was that period I went last year where I felt like everyone was just nodding in agreement that things would be nice to have, but not actually wanting to do anything about it. I'm sure it partially had to do with the fact that I was a low-priority external entity that no one really knew back then, but, it just... didn't feel all that great to be in such a meeting. Basically, you go to a meeting requesting help just to have people say go try it out by yourself, because Chisel2 is being deprecated and we're busy on Chisel3. Ok, so I did, and now all of my stuff is incompatible with stuff moving forward. What a great use of my time!

Re: DSP framework, I don't mean upstream it to Chisel. When I was working on DSP for Chisel2, the biggest issue I had was basically having to make a bleep ton of modifications to "Bits" to support adding new types, but that's not the case anymore. I think it should be a separate entity, although right now, having such a collection of different Git repos to get stuff to work is awkward.

Re: my biggest gripe as of a few months ago, I outlined a very specific reason for why I wanted heterogenous Vecs... And I think it took quite a bit of convincing [even when I first had that long discussion back in March?] that it was a useful thing... I think Jack started working on it, but I haven't followed up on it, mostly b/c I haven't been writing Chisel. I think I've been, in general, pretty good on advocating use cases. It's just that, up until recently, either people didn't care to support them or they were lower in priority. Fine, maybe the solution is that we just need more money to hire more staff engineers...

And look, it doesn't help when people say that addPin is going to be removed, and I'm out of luck otherwise. I think my reasons for wanting certain features (Optionable IO, which now exists thx; IO that doesn't need to be in IO Bundles) are reasonable, so it's frustrating to be told that, without a more convincing argument -- as in, if I can exercise it safely, why are you now preventing me from using it? Sure, maybe other people might not use it properly, but as long as the particular context in which I used it is well tested, I think that's good enough a reason to at least not take away a feature. Yay for forking?

Re: rocket-chip, I'm fine with some reliance on Makefiles, but things are just overly convoluted.

In general, I think designers should really sit down with their end users and watch how they interact with their tools, etc. There's just been an overall lack of focus on usability that really needs to change if you want things to be open-source-successful... I know it's hard b/c everyone has his/her own deadlines, but it, IMO, really needs to be done.

On a similar note, when I first was taught how to interface with rocket-chip, I actually spent a really long time writing up a how-to for getting it to work. Unfortunately, since rocket-chip's changed so much in the last half year, that became very dated very quickly...

Anyways, truth be told, I haven't been pushing stuff Chisel-related recently, b/c I'm plenty fine with some mishmash of my fully-featured Chisel2DSP and whatever was implemented in Chisel3 by the time I needed it... but I've been recently pushed to stick only with Chisel3, and it's just... missing a lot...

[stevobailey]

@chick or @shunshou Can you push a change to FIRRTL that just inlines addition of the Verilog modules? I think our discussion has diverged away from the topic at hand, and I don't think we ever settled on a better solution. Thanks!

[chick]

I'll take a look at it, but I'd like to try and figure out what's broken on fixed pointer numbers first. Chick Markley Staff Programmer -- Aspire Lab University of California, Berkeley 583-2 Soda Hall, Berkeley, CA 94720-1776 chick@berkeley.edu

…

On Thu, Jan 12, 2017 at 12:32 PM, Stevo ***@***.***> wrote: @chick <https://github.com/chick> or @shunshou <https://github.com/shunshou> Can you push a change to FIRRTL that just inlines addition of the Verilog modules? I think our discussion has diverged away from the topic at hand, and I don't think we ever settled on a better solution. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA4WV3Q2gucW1Fu8yMUgssAHcOm7BTJOks5rRo31gaJpZM4LHMGO> .

[ducky64]

@chick

Bump. It seems that there is now a way in Chisel to include Verilog with black boxes. Can this be implemented now?

[shunshou]

confirmed this works.