[CIR] Upstream missing tests from the ClangIR incubator

[andykaylor]

As we have been upstreaming code from the ClangIR incubator project, we have been creating highly targeted tests for each PR as it is reviewed. Often these tests are based on existing tests in the incubator, but in most cases at least part of the test must be omitted because it exercises a feature that is now yet supported upstream. There may also be other incubator tests which cover the new functionality which are not identified during the upstreaming process.

Over time, this is leading to a gap in depth of coverage between the incubator and the upstream implementation. It would be very helpful if someone could periodically assess the state of the incubator tests and upstream those tests which can be successfully executed with the upstream code. This will involve updating the checks in the tests to follow the upstream practices, such as testing codegen lowering to LLVM IR and adding the non-CIR LLVM IR codegen path as a point of reference. It may also be necessary to update the existing checks to account for minor differences in generated CIR.

This will be a large, ongoing task, but I believe it is suitable to someone who is new to the project. Here's how I imagine it working.

1. Clone the incubator repository, and verify that you can build and test it successfully (directions can be found here)
2. Clone the main LLVM repository, and verify that you can build and test with ClangIR enabled
3. Create a new test subdirectory, clang/test/CIR-Incubator and copy the contents of clang/test/CIR from the incubator to that directory.
4. Update the CMakeLists.txt and various lit configuration files in the new directory to create a new build target, clang-check-cir-incubator
5. Run the tests with the new target
6. Look for tests that are passing which are missing or partially missing in the upstream clang/test/CIR directory
7. Creates PRs to add these tests, updating them to upstream project standards

My description of step 4 above should be read while imagining a good bit of hand-waving. Figuring out the details of that step will be the most difficult part of this task.

I have marked this as a "good first issue" but interested contributors should be aware that this may be challenging for those who are new to the LLVM project. I believe it is a good first issue because it provides a way to gain experience with LLVM's build and test system, while also performing useful work, without requiring specific prior understanding of any part of the LLVM code base.

I (Andy Kaylor, akaylor@nvidia.com) will be happy to provide guidance to anyone willing to put in serious effort for this task.

[llvmbot]

Hi!

This issue may be a good introductory issue for people new to working on LLVM. If you would like to work on this issue, your first steps are:

1. Check that no other contributor is working on this issue. If someone is assigned to the issue or claimed to be working on it, ping the person. After one week without a response, the assignee may be changed.
2. Leave a comment indicating that you are working on the issue, or just create a pull request after following the steps below. Mention this issue in the description of the pull request.
3. Fix the issue locally.
4. Run the test suite locally. Remember that the subdirectories under test/ create fine-grained testing targets, so you can e.g. use make check-clang-ast to only run Clang's AST tests.
5. Create a Git commit.
6. Run git clang-format HEAD~1 to format your changes.
7. Open a pull request to the upstream repository on GitHub. Detailed instructions can be found in GitHub's documentation. Mention this issue in the description of the pull request.

If you have any further questions about this issue, don't hesitate to ask via a comment in the thread below.

[llvmbot]

@llvm/issue-subscribers-good-first-issue

Author: Andy Kaylor (andykaylor)

As we have been upstreaming code from the ClangIR incubator project, we have been creating highly targeted tests for each PR as it is reviewed. Often these tests are based on existing tests in the incubator, but in most cases at least part of the test must be omitted because it exercises a feature that is now yet supported upstream. There may also be other incubator tests which cover the new functionality which are not identified during the upstreaming process.

Over time, this is leading to a gap in depth of coverage between the incubator and the upstream implementation. It would be very helpful if someone could periodically assess the state of the incubator tests and upstream those tests which can be successfully executed with the upstream code. This will involve updating the checks in the tests to follow the upstream practices, such as testing codegen lowering to LLVM IR and adding the non-CIR LLVM IR codegen path as a point of reference. It may also be necessary to update the existing checks to account for minor differences in generated CIR.

This will be a large, ongoing task, but I believe it is suitable to someone who is new to the project. Here's how I imagine it working.

1. Clone the incubator repository, and verify that you can build and test it successfully (directions can be found here)
2. Clone the main LLVM repository, and verify that you can build and test with ClangIR enabled
3. Create a new test subdirectory, clang/test/CIR-Incubator and copy the contents of clang/test/CIR from the incubator to that directory.
4. Update the CMakeLists.txt and various lit configuration files in the new directory to create a new build target, clang-check-cir-incubator
5. Run the tests with the new target
6. Look for tests that are passing which are missing or partially missing in the upstream clang/test/CIR directory
7. Creates PRs to add these tests, updating them to upstream project standards

My description of step 4 above should be read while imagining a good bit of hand-waving. Figuring out the details of that step will be the most difficult part of this task.

I have marked this as a "good first issue" but interested contributors should be aware that this may be challenging for those who are new to the LLVM project. I believe it is a good first issue because it provides a way to gain experience with LLVM's build and test system, while also performing useful work, without requiring specific prior understanding of any part of the LLVM code base.

I (Andy Kaylor, akaylor@nvidia.com) will be happy to provide guidance to anyone willing to put in serious effort for this task.

I'd like to help with this task. Could you please assign it to me? Thanks!

[andykaylor]

I'd like to help with this task. Could you please assign it to me? Thanks!

I'd prefer not to assign this until I see the how the work goes. I think it may be something that multiple people could productively contribute to. Feel free to begin working on this, and if someone else wants to contribute we can figure out how to split up the work.

BTW, I think step 4 in my instructions above may be unnecessary. If the default configuration is acceptable, a default target will be created based on the directory name.

[sonar3d]

Hi I'd be interested to help out with this task if possible!

[andykaylor]

@sonar3d Thanks for offering to help! Please look at the PR @bursot has already put up for review, and read the comments there as a guide to what our expectations are for these tests. You could start by getting an environment setup and verifying that you can reproduce the tests that @brusot found to be passing. Look at the original commit in #157333, as some tests are being removed for further work, but I expect that @bursot will continue to work on them.

Other tests which require more modification before they can be upstreamed should be put in smaller PRs, with only closely related tests in a single PR. I request that when preparing PRs for tests that require modification, we should start with a commit that adds the test without modification and then add a commit which modifies the test in the same branch, with the initial PR containing both commits. This will make it easier during review to see what has changed. The commits will be squashed when the PR is merged, so this is just a temporary state to help with the review.

Let's use this GH issue as a space for coordinating the work. When you have identified a set of tests that you'd like to work on, post a comment here, tagging anyone else who is working on this task, and state that which tests you're starting work on. If this process works, we can continue that way. If not, we can arrange a chat to discuss other ways to coordinate the effort.

[kimsh02]

Hi @andykaylor , I would be interested in this task if you still require an extra hand at this point.

[sonar3d]

Hi @andykaylor , I would be interested in this task if you still require an extra hand at this point.

I am away for a few weeks from next week but keen to look at a few tests and help this weekend and when I'm back :) - happy for you to do more tasks while I'm away.

[AmrDeveloper]

Hi @kimsh02 and @sonar3d,

There are a lot of tests, and the task is still open. Feel free to start.

Some notes:

• The comments of [CIR] Upstream a batch of passing tests from CIR-Incubator #157333 contain a lot of good review points and instructions that can help you work on those tests.
• It's recommended to have a few tests per PR, so it can be easy for you to apply review comments, then once you become familiar, you can add more.
• Once you know which files you will work on, please share them here to avoid conflict with other contributors.
• If something is not clear or you need help, feel free to ask here, or on Discord

[kimsh02]

@AmrDeveloper Okay, thanks for the heads up. I will begin to take a look at things. I see the PR is quite lengthy.

[kimsh02]

Just as an update, I forked, built, and ran the tests for clang-ir today. I will need assistance on updating the CMakeLists.txt to build the main clang with the incubated clang-ir.