Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make verilator_ext_tests head-to-head #1483

Closed
veripoolbot opened this issue Jul 26, 2019 · 7 comments
Closed

Make verilator_ext_tests head-to-head #1483

veripoolbot opened this issue Jul 26, 2019 · 7 comments

Comments

@veripoolbot
Copy link
Contributor

@veripoolbot veripoolbot commented Jul 26, 2019


Author Name: Todd Strader (@toddstrader)
Original Redmine Issue: 1483 from https://www.veripool.org

Original Assignee: Todd Strader (@toddstrader)


This is still a WIP, but I think it warrants some discussion before I trudge ahead much further:
https://github.com/toddstrader/verilator_ext_tests/tree/head-to-head

I'm proposing that we have Travis always pull the latest master from every submodule (including Verilator, which I added). This way we don't have to keep bumping these submodules by hand. Right now, I'm just running Travis on my branch manually, or by pushing verilator_ext_tests. If we go with head-to-head, we can initially just turn on daily cron builds.

There's also some fanciness in Travis about public/private keys, encrypted YAML and protected environment variables. This supposedly allows you to "safely" kick off Travis builds via webhooks. I'm wondering if we could grant these projects we're building with these privileges so that they could kick off a verilator_ext_tests build when they run their own CI. This would just grant them the ability to control Travis runs and not permissions to GitHub. All of this is to say, I would need to do more research on this bit.

You'll notice that Travis is currently failing the SweRV test:
https://travis-ci.com/toddstrader/verilator_ext_tests/builds/120918474

This is actually a fortunate coincidence because the SweRV test passes if you run with the Verilator rev that is committed via the submodule (just don't update with --remote). But if you do update to HEAD of verilator, it breaks. So . . . heads up SweRV people (I'll post there).

I also learned how to cache the full Verilator build with Travis build stages. This has potential benefits for the Verilator proper CI. I'll have to circle back there. However, the solution I've landed on is a bit cumbersome. Essentially, I'd like to install Verilator with installation option #4 (from README.pod). However, this involves unsetting VERILATOR_ROOT which breaks driver.pl which these tests depend on. I'm wondering if VERILATOR_ROOT can be deprecated, but that's a topic for another thread.

What are your thoughts on this approach? It will obviously break more often. But I think that's what we want. If we have to manually bump submodule versions, I'm not sure how much more beneficial verilator_ext_tests is than these projects' own CI environments.

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Jul 26, 2019


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-07-26T22:59:39Z


I agree this is a good approach. Note Verilator, along with SwerRV is part of Chips Alliance, so if you want you can ask for commit privileges to it, and/or we can move the ext tests to the CHIPS Alliance repo if that helps in some way (e.g. they might pay for more Travis services).

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Aug 4, 2019


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-08-04T02:16:36Z


#� captures the failures due to new WIDTH warnings.

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Aug 22, 2019


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-08-22T20:13:44Z


There's more to do here, but I'm focused on the DPI protected modules for now. But I believe this should be land-able:
https://github.com/toddstrader/verilator_ext_tests/tree/head-to-head

If we turn on daily cron jobs, we'll get a head-to-head check of all these tests and Verilator every day.

Also, the build has gone red since I looked at this last:
https://travis-ci.com/toddstrader/verilator_ext_tests/builds/122567524

This looks to be a legitimate SweRV issue this time. I suggest we add a requirement to the README that external projects have their own CI using whatever their fixed version of Verilator is. We can grandfather in the current two projects and politely ask them to set up their own CI.

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Aug 27, 2019


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-08-27T22:59:20Z


These changes look fine.

Personally I'd like to be able to run the tests pointing at a separate verilator tree (I am developing at that moment) without picking up the external for verilator. Do you know if there's a way we can do that? Or maybe I just live with the external fetch and set VERILATOR_ROOT to my local area?

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Aug 28, 2019


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-08-28T09:59:09Z


That's what I've been doing. I just added ci/update_submodules.sh which does the remote submodule update for all submodules but excludes the verilator submodule if $VERILATOR_ROOT is set to something other than the in-tree copy. Does this work for your workflow?

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Aug 28, 2019


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2019-08-28T11:54:27Z


Ah, driver error. Feel free to squash and push. If you didn't already please contact the SweRV team to report their bug.

@veripoolbot
Copy link
Contributor Author

@veripoolbot veripoolbot commented Aug 28, 2019


Original Redmine Comment
Author Name: Todd Strader (@toddstrader)
Original Date: 2019-08-28T12:39:23Z


Done and done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.