Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Yosys a soft dependency of nMigen itself #371

Closed
whitequark opened this issue Apr 27, 2020 · 1 comment
Closed

Make Yosys a soft dependency of nMigen itself #371

whitequark opened this issue Apr 27, 2020 · 1 comment

Comments

@whitequark
Copy link
Member

whitequark commented Apr 27, 2020

At the moment, Yosys is a hard dependency of nMigen if back.verilog is used. Because of the debug Verilog output provided even in FOSS design flows (where the synthesis ingests RTLIL, not Verilog), nearly any practical use of nMigen beyond simulation via back.pysim requires Yosys to be installed.

Although not catastrophic (Yosys is a relatively easy to build C++ codebase), this can be a significant obstacle for beginners. Further, this forces nMigen to support a wide range of Yosys revisions (from latest stable to master, which currently spans over 4000 commits), ties nMigen to the irregular, approximately once per year Yosys release schedule, and requires Yosys-dependent transformations to gracefully degrade if they find themselves used with an outdated version.

In concrete terms, even after #324 is fixed, when is back.cxxsim "shipped"? Is it when Yosys 0.10 is released? (What if we miss the window for that and have to wait a year more for 0.11?) Is it when Yosys 0.10 is in Debian testing? Debian stable? some third party PPA? What happens if, shortly after the Yosys release, a functionality-breaking bug is discovered in the CXXRTL backend?

Morally, Yosys-as-part-of-back.verilog should not be a system dependency, but rather a part of nMigen itself. However, achieving that is not trivial.

@whitequark
Copy link
Member Author

whitequark commented Apr 27, 2020

As an experiment, I tried packaging the Yosys WASI port for PyPI using wasmtime-py. The source is available as a repository and the binary is available on TestPyPI. Run:

pip3 install wasmtime
pip3 install -i https://test.pypi.org/simple/ --pre nmigen-yosys
python3 -m nmigen_yosys

Upsides:

  • The package seems to work well on Linux and Windows. (Shouldn't have issues on macOS either, but I didn't check.)
  • The startup latency is acceptable, even in a VM.
  • The package is quite small: the compressed size is less than 500 KB, and the installed size is less than 2 MB, not including wasmtime.
  • By defining nmigen_yosys.__main__ it is possible to make /usr/bin/python3 -m nmigen_yosys a drop-in replacement for /usr/bin/yosys as long as no absolute paths are used. (nMigen doesn't use them anywhere, and further, avoids invoking Yosys with any paths at all.)

Downsides:

  • Wasmtime doesn't currently work on 32-bit x86, and python.org suggests the 32-bit installer by default. (To get to the 64-bit installer you have to follow three or four links.)
  • Wasmtime-py doesn't currently cache compiled machine code, and recompiling yosys.wasm is the majority of time spent in back.verilog on small and medium size designs.
  • The startup latency (especially given the previous downside) and package size are acceptable only because I've committed some crimes. Yosys does not allow choosing an arbitrary set of passes to be built (in fact it does not even track pass dependencies), and the SMALL:=1 binaries are not so small (about 10 MB stripped), so it is necessary to override the list of targets that go into the final executable.
  • WASI is not yet upstream in Yosys.

Overall, this solution seems acceptable; the main problem is the lack of 32-bit Windows wheels in wasmtime. By using python3 -m nmigen_yosys as a drop-in replacement, the Yosys packaging strategy can be changed to anything else at a later date, so the risks of going with WASI are low; worst case, we just have to go back to building win32 and anylinux wheels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

1 participant