Link against CIRCT and use it for codegen #234

fabianschuiki · 2021-11-21T11:23:47Z

This PR makes Moore link against the circt library and uses that to generate code.

More specifically, this adds a circt-sys crate with the low-level bindings against the MLIR C library (generated by the bindgen crate), and a circt crate which contains some higher-level wrapper code that makes CIRCT and MLIR easier to use from Rust code. Note that we do not establish any kind of safety in Rust terms, but the API is "safe enough" for what we're trying to use it for.

Currently codegen.rs calls the original LLHD crate and the new CIRCT bindings in parallel to generate code. This was useful as a sanity check and to make sure the CIRCT-based emission is up to par with the LLHD crate. At a later point we should strip codegen down to use only CIRCT.

This PR also adds a --format=mlir-native output option which uses MLIR's dialect printer to emit the assembly.

Add a first bindgen-based `circt-sys` crate that allows Rust code to use the CIRCT and, by extension, MLIR project code.

Add a `circt` crate that contains a lightweight wrapper to make `circt-sys` more comfortable to use from the Rust world. Add this crate as a dependency of `svlog`, for later use in module emission.

fabianschuiki · 2021-11-21T11:25:51Z

@maerhart I'd love to get your feedback/review on this, since you also opened issue #232 😃

maerhart

Excellent work! This is a big step forward in terms of merging Moore with CIRCT.
I see you even incorporated FileCheck tests.

Just a few comments, mostly on index bitwidth and unique name checking.

Additionally, it could be useful to add some instructions to the README on how to build moore as it got quite a bit more complicated with CIRCT as a submodule (or did I miss some automation?).
E.g.,

git submodule update --init --recursive
mkdir -p circt/build
mkdir -p circt/llvm/build
cd circt/llvm/build
cmake ../llvm \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=../install \
    -DLLVM_BUILD_EXAMPLES=OFF \
    -DLLVM_ENABLE_ASSERTIONS=ON \
    -DLLVM_ENABLE_BINDINGS=OFF \
    -DLLVM_ENABLE_OCAMLDOC=OFF \
    -DLLVM_ENABLE_PROJECTS=mlir \
    -DLLVM_INSTALL_UTILS=ON \
    -DLLVM_OPTIMIZED_TABLEGEN=ON \
    -DLLVM_TARGETS_TO_BUILD=""
cmake --build . --target install
cd ../../build
cmake .. -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_INSTALL_PREFIX=../install \
    -DMLIR_DIR=$PWD/../llvm/install/lib/cmake/mlir \
    -DLLVM_DIR=$PWD/../llvm/install/lib/cmake/llvm \
    -DLLVM_ENABLE_ASSERTIONS=ON
cmake --build . --target install
cd ../..
export CIRCT_SYS_CIRCT_DIR=`pwd`/circt
export CIRCT_SYS_CIRCT_BUILD_DIR=`pwd`/circt/install
export CIRCT_SYS_LLVM_DIR=`pwd`/circt/llvm
export CIRCT_SYS_LLVM_BUILD_DIR=`pwd`/circt/llvm/install
cargo build

.github/build-circt.sh

.github/workflows/ci.yml

src/circt-sys/build.rs

src/circt/src/comb.rs

maerhart · 2021-11-22T14:57:42Z

src/circt/src/hw.rs

+        offset: usize,
+        length: usize,
+    ) -> Self {
+        let offset = crate::hw::ConstantOp::new(builder, 64, &offset.into()).into();


The IR emitted here will fail during the CIRCT verification pass because ArraySliceOp expects the bitwidth of offset to be llvm::Log2_64_Ceil(type_cast<ArrayType>($_self).getSize()).

That's an excellent catch! I should have added a run of the verifier into Moore directly, just after codegen. Let me do that.

Yes, that's a good idea!

src/svlog/codegen.rs

maerhart · 2021-11-22T17:11:02Z

src/svlog/codegen.rs

-                        self.builder.ins().sig(v)
+                        (
+                            self.builder.ins().sig(v.0),
+                            circt::llhd::SignalOp::new(self.mlir_builder, "", v.1).into(),


Same as above.

maerhart · 2021-11-22T17:16:05Z

src/svlog/codegen.rs

+            circt::llhd::VariableOp::new(self.mlir_builder, inner).into()
+        } else if circt::llhd::is_signal_type(ty) {
+            let inner = self.emit_zero_for_type_mlir(circt::llhd::signal_type_element(ty));
+            circt::llhd::SignalOp::new(self.mlir_builder, "", inner).into()


Same as above.

maerhart · 2021-11-22T17:18:22Z

src/svlog/codegen.rs

+                let init = self.emit_zero_for_type_both(ty);
+                let sig = (
+                    self.builder.ins().sig(init.0),
+                    circt::llhd::SignalOp::new(self.mlir_builder, "", init.1).into(),


Same as above.

maerhart · 2021-11-22T17:37:10Z

src/svlog/codegen.rs

-                    self.builder.ins().br_cond(event_cond, init_blk, event_blk);
-                    self.builder.append_to(event_blk);
+                    let event_blk = self.mk_block(Some("event"));
+                    self.mk_cond_br(event_cond, init_blk, event_blk);


This is the only call to mk_cond_br where the successors are not transposed. Are you sure that init_blk and event_blk shouldn't be transposed?

Ufff that would have been annoying to debug. Thanks a lot! 😬

fabianschuiki · 2021-11-23T07:01:14Z

Thanks a lot for the extremely detailed review @maerhart! This is awesome, and you caught quite a few bugs 😬.

Additionally, it could be useful to add some instructions to the README on how to build moore as it got quite a bit more complicated with CIRCT as a submodule (or did I miss some automation?).

Yeah that is a good idea. I'll add the snippet to the README. I was considering making the circt-sys crate fetch and build LLVM and CIRCT automatically if you don't provide the environment variables, but not sure if a 2 hour cargo check is what the user would expect. Maybe it's easier to just provide good instructions to the user, as you suggest?

maerhart · 2021-11-23T11:23:38Z

Maybe it's easier to just provide good instructions to the user, as you suggest?

I'd say good instructions are good enough for now. Automatic fetch and build is certainly more convenient from a user perspective (provided it always succeeds), but it should definitely print in the CLI when it does so and report the progress, otherwise one might think it ran into an infinite loop or so 😆

Some tools like `llvm-lit` rely on the build directory and project source tree to be available, so it makes sense to make the CI look like it was a regular development build that you would do on your own machine.

fabianschuiki added 30 commits November 13, 2021 19:31

mlir: Add a basic circt-sys crate

bd1e194

Add a first bindgen-based `circt-sys` crate that allows Rust code to use the CIRCT and, by extension, MLIR project code.

mlir: Add higher-level circt crate

7621f92

Add a `circt` crate that contains a lightweight wrapper to make `circt-sys` more comfortable to use from the Rust world. Add this crate as a dependency of `svlog`, for later use in module emission.

mlir: Add LLHD and standard dialect, restructure

a80e56e

mlir: Add first codegen support

caca0d1

mlir: Move circt crates into src directory

8765141

mlir: Add CIRCT submodule and build script

65d8b0a

mlir: Compile order fixes

a84e8c8

mlir: Implement operation printing to a Write

f2633e5

mlir: Add LLVM lit-based tests

c0d27ec

mlir: Add helper to go from MlirStringRef to &str

1ff86d4

mlir: Move ModuleOp into a new builtin module

bbe43a8

mlir: Add support for debug info and location translation

ca620d5

mlir: Add MLIR type emission

983e027

mlir: Add LLHD dialect type support

ee28b3c

mlir: Emit proper entity type signatures

639e56d

mlir: First expression emissions

dda21c6

mlir: Add first declaration support

f509127

mlir: Add port default value driving

bc806e6

mlir: Basic process emission

ff32e20

mlir: Add process prolog/epilog

864a88f

mlir: Impl rvalue codegen

32bb626

mlir: Update CI to include CIRCT/LLVM and lit tests

e93cf19

mlir: Add missing struct/array const emission

0390669

mlir: Get new codegen to not crash for all tests

da9ffec

mlir: Stop using llhd.not and llhd.neg ops

730af9d

mlir: Add module and process instantiation

4385ed6

mlir: Impl if stmt codegen

c4c62c8

mlir: Impl loop codegen

ec8c5cd

mlir: Impl procedural assignment codegen

cda34a0

mlir: Impl timed wait codegen

15d2982

mlir: Remove obsolete codegen functions

7c99782

fabianschuiki added L-vlog Language: Verilog and SystemVerilog. C-enhancement Category: Adding or improving on features. A-codegen Area: Code generation. labels Nov 21, 2021

This was referenced Nov 21, 2021

Add Moore MIR dialect and move codegen to CIRCT #235

Open

Remove legacy LLHD crate codegen and dependency #236

Open

maerhart reviewed Nov 22, 2021

View reviewed changes

fabianschuiki added 3 commits November 24, 2021 14:59

mlir: Update circt submodule

4b53323

mlir: Update README with CIRCT/LLVM/MLIR instructions

68651cb

mlir: Point CI build to install directories instead of build

5a35b81

fabianschuiki force-pushed the mlir branch from 165e304 to 5a35b81 Compare November 24, 2021 14:16

fabianschuiki added 13 commits November 24, 2021 15:21

mlir: Drop xml2 from CIRCT system library list

57f9f32

mlir: Report MLIR diagnostics through Moore's diagnostic engine

7d32f19

mlir: Remove unused ops, fix cond_br, fix typos

eb13fdb

mlir: Pick a dummy name for process instances

591acfa

mlir: Revert CI back to using build dir, don't install

33a5b1f

Some tools like `llvm-lit` rely on the build directory and project source tree to be available, so it makes sense to make the CI look like it was a regular development build that you would do on your own machine.

mlir: Fix ICmpOp return type and ExtractOp attr type

540e431

mlir: Uniquify signal and instance names

bab3cf7

mlir: Improve MLIR location tracking

09cf0e0

mlir: Improve circt-sys rebuild on CIRCT recompilation

874a108

mlir: Properly truncate indices/offsets to clog2 of array/integer size

1a88094

mlir: Fix minor emission issues

1730612

mlir: Speed up location translations

5416d84

rustfmt: Fix vhdl/score/lower_hir.rs

de5abbb

fabianschuiki merged commit f1aaa9c into master Jan 16, 2022

fabianschuiki deleted the mlir branch January 16, 2022 08:40

fabianschuiki mentioned this pull request Jan 16, 2022

Add CIRCT HW/Comb/LLHD as backend #232

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Link against CIRCT and use it for codegen #234

Link against CIRCT and use it for codegen #234

fabianschuiki commented Nov 21, 2021

fabianschuiki commented Nov 21, 2021 •

edited

maerhart left a comment

maerhart Nov 22, 2021

fabianschuiki Nov 23, 2021

maerhart Nov 23, 2021

maerhart Nov 22, 2021

maerhart Nov 22, 2021

maerhart Nov 22, 2021

maerhart Nov 22, 2021

fabianschuiki Nov 23, 2021

fabianschuiki commented Nov 23, 2021

maerhart commented Nov 23, 2021

Link against CIRCT and use it for codegen #234

Link against CIRCT and use it for codegen #234

Conversation

fabianschuiki commented Nov 21, 2021

fabianschuiki commented Nov 21, 2021 • edited

maerhart left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabianschuiki commented Nov 23, 2021

maerhart commented Nov 23, 2021

fabianschuiki commented Nov 21, 2021 •

edited