[utils][UpdateLLCTestChecks] Add MIR support to update_llc_test_checks.py.

[vpykhtin]

This change enables update_llc_test_checks.py to automatically generate MIR checks for RUN lines that use -stop-before or -stop-after flags allowing tests to verify intermediate compilation stages (e.g., after instruction selection but before peephole optimizations) alongside the final assembly output.

This resulted from the scenario, when I needed to test two instruction matching patterns where the later pattern in the peepholler reverts the earlier pattern in the instruction selector and distinguish it from the case when the earlier pattern didn't worked at all.

Drawback: this may create very noisy tests.

Note: This is 100% Claude Sonnet 4.5 and I don't feel I understand the change in details, but the change looks so small that I decided to give it a chance.

Example:

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
; RUN: llc -mtriple=amdgcn -mcpu=gfx90a < %s | FileCheck -check-prefix=GFX90A %s
; RUN: llc -mtriple=amdgcn -mcpu=gfx90a -stop-before=si-conditional-fma-peephole < %s | FileCheck -check-prefix=MIR %s

; Pattern 1: Direct fsub with select
; result = a - (cond ? c : 0.0)
define double @cond_sub_basic(double %a, double %c, i1 %cond) {
; GFX90A-LABEL: cond_sub_basic:
; GFX90A:       ; %bb.0:
; GFX90A-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX90A-NEXT:    v_and_b32_e32 v4, 1, v4
; GFX90A-NEXT:    v_cmp_eq_u32_e32 vcc, 1, v4
; GFX90A-NEXT:    v_cndmask_b32_e32 v2, 0, v2, vcc
; GFX90A-NEXT:    v_cndmask_b32_e32 v3, 0, v3, vcc
; GFX90A-NEXT:    v_add_f64 v[0:1], v[0:1], -v[2:3]
; GFX90A-NEXT:    s_setpc_b64 s[30:31]
  ; MIR-LABEL: name: cond_sub_basic
  ; MIR: bb.0 (%ir-block.0):
  ; MIR-NEXT:   liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4
  ; MIR-NEXT: {{  $}}
  ; MIR-NEXT:   [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr4
  ; MIR-NEXT:   undef [[COPY1:%[0-9]+]].sub1:vreg_64_align2 = COPY $vgpr3
  ; MIR-NEXT:   [[COPY1:%[0-9]+]].sub0:vreg_64_align2 = COPY $vgpr2
  ; MIR-NEXT:   undef [[COPY2:%[0-9]+]].sub1:vreg_64_align2 = COPY $vgpr1
  ; MIR-NEXT:   [[COPY2:%[0-9]+]].sub0:vreg_64_align2 = COPY $vgpr0
  ; MIR-NEXT:   [[V_AND_B32_e32_:%[0-9]+]]:vgpr_32 = V_AND_B32_e32 1, [[COPY]], implicit $exec
  ; MIR-NEXT:   [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 -1074790400, implicit $exec
  ; MIR-NEXT:   [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 1, [[V_AND_B32_e32_]], implicit $exec
  ; MIR-NEXT:   undef [[V_CNDMASK_B32_e64_:%[0-9]+]].sub1:vreg_64_align2 = V_CNDMASK_B32_e64 0, 0, 0, [[V_MOV_B32_e32_]], [[V_CMP_EQ_U32_e64_]], implicit $exec
  ; MIR-NEXT:   [[V_CNDMASK_B32_e64_:%[0-9]+]].sub0:vreg_64_align2 = AV_MOV_B32_IMM_PSEUDO 0, implicit $exec
  ; MIR-NEXT:   [[COPY2:%[0-9]+]]:vreg_64_align2 = nofpexcept V_FMAC_F64_e32 [[V_CNDMASK_B32_e64_]], [[COPY1]], [[COPY2]], implicit $mode, implicit $exec
  ; MIR-NEXT:   $vgpr0 = COPY [[COPY2]].sub0
  ; MIR-NEXT:   $vgpr1 = COPY [[COPY2]].sub1
  ; MIR-NEXT:   SI_RETURN implicit $vgpr0, implicit $vgpr1
  %sel = select i1 %cond, double %c, double 0.0
  %result = fsub double %a, %sel
  ret double %result
}

[llvmbot]

@llvm/pr-subscribers-testing-tools

Author: Valery Pykhtin (vpykhtin)

Changes

This change enables update_llc_test_checks.py to automatically generate MIR checks for RUN lines that use -stop-before or -stop-after flags allowing tests to verify intermediate compilation stages (e.g., after instruction selection but before peephole optimizations) alongside the final assembly output.

This resulted from the scenario, when I needed to test two instruction matching patterns where the later pattern in the peepholler reverts the earlier pattern in the instruction selector and distinguish it from the case when the earlier pattern didn't worked at all.

Drawback: this may create very noisy tests.

Note: This is 100% Claude Sonnet 4.5 and I don't feel I understand the change in details, but the change looks so small that I decided to give it a chance.

---

Full diff: https://github.com/llvm/llvm-project/pull/164965.diff

1 Files Affected:

• (modified) llvm/utils/update_llc_test_checks.py (+48-7)

diff --git a/llvm/utils/update_llc_test_checks.py b/llvm/utils/update_llc_test_checks.py
index 8c57e75f34f75..5c008e2d2ed2c 100755
--- a/llvm/utils/update_llc_test_checks.py
+++ b/llvm/utils/update_llc_test_checks.py
@@ -13,9 +13,11 @@
 from traceback import print_exc
 import argparse
 import os  # Used to advertise this file's name ("autogenerated_note").
+import re
 import sys
 
 from UpdateTestChecks import common
+import update_mir_test_checks # Reuse MIR parsing code.
 
 # llc is the only llc-like in the LLVM tree but downstream forks can add
 # additional ones here if they have them.
@@ -23,6 +25,9 @@
     "llc",
 ]
 
+def has_stop_pass_flag(llc_args):
+    """Check if llc arguments contain -stop-before or -stop-after flag."""
+    return re.search(r'-stop-(?:before|after)=', llc_args) is not None
 
 def update_test(ti: common.TestInfo):
     triple_in_ir = None
@@ -119,6 +124,10 @@ def update_test(ti: common.TestInfo):
         ginfo=ginfo,
     )
 
+    # Dictionary to store MIR function bodies separately
+    mir_func_dict = {}
+    mir_run_list = []
+
     for (
         prefixes,
         llc_tool,
@@ -141,14 +150,31 @@ def update_test(ti: common.TestInfo):
         if not triple:
             triple = common.get_triple_from_march(march_in_cmd)
 
-        scrubber, function_re = output_type.get_run_handler(triple)
-        if 0 == builder.process_run_line(
-            function_re, scrubber, raw_tool_output, prefixes
-        ):
-            common.warn(
-                "Couldn't match any function. Possibly the wrong target triple has been provided"
+        # Check if this run generates MIR output
+        if has_stop_pass_flag(llc_args):
+            common.debug("Detected MIR output mode for prefixes:", str(prefixes))
+            # Initialize MIR func_dict for these prefixes.
+            for prefix in prefixes:
+                if prefix not in mir_func_dict:
+                    mir_func_dict[prefix] = {}
+            
+            # Build MIR function dictionary using imported function.
+            update_mir_test_checks.build_function_info_dictionary(
+                ti.path, raw_tool_output, triple, prefixes, mir_func_dict, ti.args.verbose
             )
-        builder.processed_prefixes(prefixes)
+            
+            # Store this run info for later.
+            mir_run_list.append((prefixes, llc_tool, llc_args, triple_in_cmd, march_in_cmd))
+        else:
+            # Regular assembly output.
+            scrubber, function_re = output_type.get_run_handler(triple)
+            if 0 == builder.process_run_line(
+                function_re, scrubber, raw_tool_output, prefixes
+            ):
+                common.warn(
+                    "Couldn't match any function. Possibly the wrong target triple has been provided"
+                )
+            builder.processed_prefixes(prefixes)
 
     func_dict = builder.finish_and_get_func_dict()
     global_vars_seen_dict = {}
@@ -221,6 +247,21 @@ def update_test(ti: common.TestInfo):
                         is_filtered=builder.is_filtered(),
                     )
                 )
+                
+                # Also add MIR checks if we have them for this function
+                if mir_run_list and func_name:
+                    common.add_mir_checks_for_function(
+                        ti.path,
+                        output_lines,
+                        mir_run_list,
+                        mir_func_dict,
+                        func_name,
+                        single_bb=False, # Don't skip basic block labels.
+                        print_fixed_stack=False, # Don't print fixed stack (ASM tests don't need it).
+                        first_check_is_next=False, # First check is LABEL, not NEXT.
+                        at_the_function_name=False, # Use "name:" not "@name".
+                    )
+                
                 is_in_function_start = False
 
             if is_in_function:

[github-actions]

✅ With the latest revision this PR passed the Python code formatter.

[vpykhtin]

Ok, this has failed testing, going to fix and add new one.

[arichardson]

Could you move the functions you used here into common instead of importing from the script?

[vpykhtin]

Thanks, done. Probably would be nice to move MIR specific code to the new mir.py, part of MIR code was already moved into common in #140296.

[vpykhtin]

I decided to separate the move into a separate commit and rebase this PR on top of it.

[arichardson]

Two minor comments otherwise LGTM. Thanks

[mtrofin]

Maybe naive question, but for your scenario, can you run just the passes you need, on MIR? I.e you could start with the pattern that you expect to be undone, and not rely on the earlier pass producing it. You'd also not need this change then.

... which would avoid teasing the sentiments in this thread:

https://discourse.llvm.org/t/rfc-llvm-ai-tool-policy-start-small-no-slop/88476

(I'm assuming @arichardson's lgtm means at least one person is ok with maintaining Claude's output here)

[vpykhtin]

Maybe naive question, but for your scenario, can you run just the passes you need, on MIR? I.e you could start with the pattern that you expect to be undone, and not rely on the earlier pass producing it. You'd also not need this change then

I definitely can do this, and also I can add manually created checks in my scenario (like grepping for a specific instruction presence), we already have similar test in our code base.

Ok, let it sit here for a while. Probably the main question for me here if such functionality is desired at all.

[arichardson]

Maybe naive question, but for your scenario, can you run just the passes you need, on MIR? I.e you could start with the pattern that you expect to be undone, and not rely on the earlier pass producing it. You'd also not need this change then.

... which would avoid teasing the sentiments in this thread:

https://discourse.llvm.org/t/rfc-llvm-ai-tool-policy-start-small-no-slop/88476

(I'm assuming @arichardson's lgtm means at least one person is ok with maintaining Claude's output here)

The reason I approved this is that the change looks trivial and I have had cases in the past where I wanted to test the llc output both after a specific pass (usually ISel) as well as the final assembly. Not having to have two tests that could get out of sync is quite nice there but of course you could also use manual check lines or add a MIR test.

[rnk]

This PR actually came up at the LLVM developer meeting round table on AI generated contributions (notes), and I said this sounds like a totally appropriate use for LLMs. The underlying principle behind the policy has been "how do we ensure that users are not wasting maintainer time by offloading the work of reviewing LLM generated code onto maintainers?"

When it comes to Python utility scripts in the repository, I don't think that maintainers are going through these scripts with a fine-toothed comb to ensure that they pose no threat to the correctness and stability of LLVM compilers, so I think we can say clearly that this is not wasting maintainer time and is an encouraged of LLMs.

[vpykhtin]

Rebased and squashed.

After some testing I made additional changes:

• Modified code to make cleaner separation between run_list and mir_run_list.
• Added code and test to check conflicts between ASM and MIR prefixes.
• Added test on update of already updated source (I got double checks at some point).

Check indentation looks consistent with ASM checks.

[arsenm]

Use opaque pointers

[arsenm]

Opaque pointers