Generating build files with cmake #21

vguerra · 2023-09-27T20:41:57Z

No description provided.

Using git submodules.

Adding the LLVM module so that all `lit` related functions are available. A new target is made available for running tests: `check-mlir-tutorial`.

So that `tutorial-opt` can be built. This introduces as well a new target `tutorial-opt`.

This introduces a new mlir lib: `AffineFullUnroll` and a new build target to build it: `MLIRAffineFullUnrollPasses`.

to have access to all tablgen related cmake statements.

This introduces a new mlir lib: `MulToAdd` and a new build target to build it: `MLIRMulToAddPasses`.

So that on the cmake side of things we can use the `add_mlir_dialect` macro without too many changes to the include directives. Ref: https://github.com/llvm/llvm-project/blob/main/mlir/cmake/modules/AddMLIR.cmake#L166-L177

vguerra · 2023-09-27T20:48:36Z

This is an attempt to build the tutorial generating build files with CMake. I tried to make minimal changes to the source code itself, but given some CMake helper functions, notably add_mlir_dialect, renaming PolyOps.td to Poly.td would be less intrusive.

I have added as well a github action to test building using CMake (ninja).

Let me know what you think about this PR. I could break things up. If people want to have access to how to build things up from scratch there is the entire commit history at mlir-tutorial-cmake

j2kun · 2023-09-29T03:40:50Z

Thank you for this! I will have to spend some time studying it to make sure I understand it.

j2kun · 2023-09-29T03:44:41Z

In re add_mlir_dialect, that is such a weird upstream convention. I wonder if they would consider accepting a patch to make it more flexible... but otherwise, it doesn't seem that bad to just inline that particular function to customize the names.

vguerra · 2023-09-29T09:14:14Z

Thank you for this! I will have to spend some time studying it to make sure I understand it.

Sure, take your time to go through the changes. Let me know if anything is not clear.

In re add_mlir_dialect, that is such a weird upstream convention. I wonder if they would consider accepting a patch to make it more flexible... but otherwise, it doesn't seem that bad to just inline that particular function to customize the names.

Indeed ... one can definitely use directly the mlir_tablegen command to have control on resulting *inc files. Will look into that :).

Instead of using `add_mlir_dialect` helper function for dialect generation, we inline it here so that we can have control over the naming of generated `*.inc` files. This avoids renaming of `PolyOps.td` so that cmake builds work well on the existing source code.

vguerra · 2023-09-29T11:56:25Z

Indeed ... one can definitely use directly the mlir_tablegen command to have control on resulting *inc files. Will look into that :).

addressed this in c44cb50

fruffy · 2023-10-10T21:13:08Z

Very useful, thanks! Is this ready to be used or are there still things missing?

vguerra · 2023-10-11T08:44:34Z

Is this ready to be used or are there still things missing?

This is ready to be used yes ... the only thing to keep in mind is that if #20 gets merged I might need to update this

j2kun · 2023-10-11T17:07:40Z

I promise, I am going to merge this soon :) I've just been busy traveling for work. Currently at the LLVM developers meeting, where everyone is telling me how much they enjoy these tutorials.

vguerra · 2023-10-11T18:54:14Z

where everyone is telling me how much they enjoy these tutorials.

+1 for that, this series of tutorials is very instructive and useful.

j2kun

It looks like the CI job hit an error at the end: https://github.com/j2kun/mlir-tutorial/pull/21/checks under "Post run actions." Any idea what that's about?

Post job cleanup.
/usr/bin/git version
git version [2](https://github.com/j2kun/mlir-tutorial/actions/runs/6351179046/job/17251997272#step:14:2).[4](https://github.com/j2kun/mlir-tutorial/actions/runs/6351179046/job/17251997272#step:14:4)2.0
Temporarily overriding HOME='/home/runner/work/_temp/df828214-0ad7-4d0b-b9[6](https://github.com/j2kun/mlir-tutorial/actions/runs/6351179046/job/17251997272#step:14:6)f-aaf1[7](https://github.com/j2kun/mlir-tutorial/actions/runs/6351179046/job/17251997272#step:14:7)a9c[8](https://github.com/j2kun/mlir-tutorial/actions/runs/6351179046/job/17251997272#step:14:8)6b6' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
/usr/bin/git config --global --add safe.directory /home/runner/work/mlir-tutorial/mlir-tutorial
/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
fatal: not a git repository: externals/llvm-project/../../.git/modules/externals/llvm-project
Warning: The process '/usr/bin/git' failed with exit code 128

j2kun · 2023-10-16T03:40:07Z

.gitmodules

It would be worthwhile to have the git submodule sync with the commit in bazel/import_llvm.bzl, but it doesn't appear to be possible to have git do that in this config file. It only supports a branch. Maybe after merging this I will add a note to the readme for how to do this, with something like

cd externals/llvm-project git checkout -b upstream $(grep LLVM_COMMIT ../../bazel/import_llvm.bzl | grep -o '".*"' | sed 's/"//g') cd ../..

it would be indeed nice to be able to keep in sync the llvm commit for both build systems, I guess a note as you propose would be enough as i dont think this will change often.

As a side note, I remember using a llvm checkout from september when i started to do the Cmake work and there were already some build errors due to some changes in MLIR.

j2kun · 2023-10-16T04:02:17Z

.github/workflows/build_and_test_cmake.yml

+      - name: Build and test mlir-tutorial
+        run: |
+          mkdir build && cd build
+          cmake -DLLVM_DIR=$PWD/../externals/llvm-project/build/lib/cmake/llvm -DMLIR_DIR=$PWD/../externals/llvm-project/build/lib/cmake/mlir ..


nit: might be better to use $github_workspace/externals/... instead of the relative version

indeed, will change that.

addressed in #24

j2kun · 2023-10-16T04:03:39Z

LGTM! Will give you a day or two to respond to the above, then merge either way, and I can clean up the nits.

j2kun · 2023-10-16T07:40:26Z

Eh, I'll just merge it. Thanks!!!

vguerra · 2023-10-16T15:30:20Z

It looks like the CI job hit an error at the end: https://github.com/j2kun/mlir-tutorial/pull/21/checks under "Post run actions." Any idea what that's about?

Not sure what this errors means but will have a look at that and report back.

vguerra · 2023-10-16T15:30:41Z

thanks for having a look and merging

vguerra · 2023-10-23T08:48:47Z

It looks like the CI job hit an error at the end: https://github.com/j2kun/mlir-tutorial/pull/21/checks under "Post run actions." Any idea what that's about?

Not sure what this errors means but will have a look at that and report back.

addressed in #24

vguerra added 14 commits September 27, 2023 22:18

ignoring cmake related build files

b4264ce

Adding LLVM dependency as external dependecy.

641f13e

Using git submodules.

Initial workspace setup with CMake

e4b100f

Including tests into the CMake build.

43b8cec

Adding the LLVM module so that all `lit` related functions are available. A new target is made available for running tests: `check-mlir-tutorial`.

Including tools directory into the CMake build

e9afcd6

So that `tutorial-opt` can be built. This introduces as well a new target `tutorial-opt`.

Adding affine full unroll related passes to cmake build

6a781af

This introduces a new mlir lib: `AffineFullUnroll` and a new build target to build it: `MLIRAffineFullUnrollPasses`.

Adding tablgen cmake module

10b074a

to have access to all tablgen related cmake statements.

add mlir include dirs

95316ed

Adding mul to add related passes to cmake build

7b7de0a

This introduces a new mlir lib: `MulToAdd` and a new build target to build it: `MLIRMulToAddPasses`.

Renaming PolyOps.td to Poly.td

0b0f570

So that on the cmake side of things we can use the `add_mlir_dialect` macro without too many changes to the include directives. Ref: https://github.com/llvm/llvm-project/blob/main/mlir/cmake/modules/AddMLIR.cmake#L166-L177

ignoring vs code metafiles plus compilation database files.

7a0c5b1

Adding poly dialect lib to cmake build

6564842

Adding CI for cmake build.

2b69d46

Adding canonicalizers to cmake build

38b7102

j2kun reviewed Oct 16, 2023

View reviewed changes

j2kun merged commit d04fa9c into j2kun:main Oct 16, 2023
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generating build files with cmake #21

Generating build files with cmake #21

vguerra commented Sep 27, 2023

vguerra commented Sep 27, 2023

j2kun commented Sep 29, 2023

j2kun commented Sep 29, 2023 •

edited

vguerra commented Sep 29, 2023

vguerra commented Sep 29, 2023

fruffy commented Oct 10, 2023

vguerra commented Oct 11, 2023

j2kun commented Oct 11, 2023

vguerra commented Oct 11, 2023

j2kun left a comment

j2kun Oct 16, 2023

vguerra Oct 16, 2023

j2kun Oct 16, 2023

vguerra Oct 16, 2023

vguerra Oct 23, 2023

j2kun commented Oct 16, 2023

j2kun commented Oct 16, 2023

vguerra commented Oct 16, 2023

vguerra commented Oct 16, 2023

vguerra commented Oct 23, 2023

Generating build files with cmake #21

Generating build files with cmake #21

Conversation

vguerra commented Sep 27, 2023

vguerra commented Sep 27, 2023

j2kun commented Sep 29, 2023

j2kun commented Sep 29, 2023 • edited

vguerra commented Sep 29, 2023

vguerra commented Sep 29, 2023

fruffy commented Oct 10, 2023

vguerra commented Oct 11, 2023

j2kun commented Oct 11, 2023

vguerra commented Oct 11, 2023

j2kun left a comment

Choose a reason for hiding this comment

j2kun Oct 16, 2023

Choose a reason for hiding this comment

vguerra Oct 16, 2023

Choose a reason for hiding this comment

j2kun Oct 16, 2023

Choose a reason for hiding this comment

vguerra Oct 16, 2023

Choose a reason for hiding this comment

vguerra Oct 23, 2023

Choose a reason for hiding this comment

j2kun commented Oct 16, 2023

j2kun commented Oct 16, 2023

vguerra commented Oct 16, 2023

vguerra commented Oct 16, 2023

vguerra commented Oct 23, 2023

j2kun commented Sep 29, 2023 •

edited