[Dexter] Extend XFAIL of Dexter tests to all MacOS architectures.

[dyung]

I am trying to bring up a MacOS buildbot targeting x86 and noticed that two Dexter tests were failing, cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp and cross-project-tests/debuginfo-tests/llgdb-tests/static-member-2.cpp. Looking in the history for these tests, they were XFAILed for Apple Silicon in 9c46606. To test, I locally removed the XFAIL on my local Apple Silicon machine, and the errors produced match those my buildbot on x86 is producing.

For example:

FAIL: cross-project-tests :: debuginfo-tests/llgdb-tests/static-member.cpp (132 of 137)
******************** TEST 'cross-project-tests :: debuginfo-tests/llgdb-tests/static-member.cpp' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
Debugger output was:
error: 'delete' is not a valid command.
imported lldb from: "/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python"
>  delete breakpoints
None
>  break static-member.cpp:33
SBBreakpoint: id = 1, file = 'static-member.cpp', line = 33, exact_match = 0, locations = 1
>  r
success
>  ptype MyClass
class MyClass {
    static const int a = 4;
    static int b;
    static int c;
public:
    int d;
}
>  p MyClass::a
(const int) $0 = 4
>  p MyClass::c
(int) $1 = 15
> quit

--
Command Output (stderr):
--
RUN: at line 1: /Users/dyung/src/git/llvm/build/bin/clang --driver-mode=g++ --target=arm64-apple-darwin23.3.0 -O0 -g /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp -o /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp -c
+ /Users/dyung/src/git/llvm/build/bin/clang --driver-mode=g++ --target=arm64-apple-darwin23.3.0 -O0 -g /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp -o /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp -c
RUN: at line 2: /Users/dyung/src/git/llvm/build/bin/clang --driver-mode=g++ --target=arm64-apple-darwin23.3.0 /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp -o /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp.out
+ /Users/dyung/src/git/llvm/build/bin/clang --driver-mode=g++ --target=arm64-apple-darwin23.3.0 /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp -o /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp.out
RUN: at line 3: /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/test_debuginfo.pl /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp.out
+ /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/test_debuginfo.pl /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.tmp.out
/Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp:11:11: error: CHECK: expected string not found in input
// CHECK: static const int a;
          ^
/Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.gdb.output:10:16: note: scanning from here
class MyClass {
               ^
/Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.gdb.output:11:2: note: possible intended match here
 static const int a = 4;
 ^

Input file: /Users/dyung/src/git/llvm/build/projects/cross-project-tests/debuginfo-tests/llgdb-tests/Output/static-member.cpp.gdb.output
Check file: /Users/dyung/src/git/llvm/cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            .
            .
            .
            5: > break static-member.cpp:33 
            6: SBBreakpoint: id = 1, file = 'static-member.cpp', line = 33, exact_match = 0, locations = 1 
            7: > r 
            8: success 
            9: > ptype MyClass 
           10: class MyClass { 
check:11'0                    X error: no match found
           11:  static const int a = 4; 
check:11'0     ~~~~~~~~~~~~~~~~~~~~~~~~~
check:11'1      ?                        possible intended match
           12:  static int b; 
check:11'0     ~~~~~~~~~~~~~~~
           13:  static int c; 
check:11'0     ~~~~~~~~~~~~~~~
           14: public: 
check:11'0     ~~~~~~~~
           15:  int d; 
check:11'0     ~~~~~~~~
           16: } 
check:11'0     ~~
            .
            .
            .
>>>>>>

This matches what my x86 MacOS buildbot seems to be hitting for this test https://lab.llvm.org/staging/#/builders/188/builds/307/steps/6/logs/FAIL__cross-project-tests__static-member_cpp.

The failing output for static-member-2.cpp also seems to match between my Apple Silicon and x86 machines, so I think it makes sense to extend the XFAIL to all system-darwin targets, and not just aarch64.

[JDevlieghere]

We run the debug info tests on GreenDragon on an Intel machine, so if they're not currently XFAIL'd they should be passing there (I'd double check but green.lab.llvm.org is having trouble currently). That said, if they're already disabled on arm I don't mind extending this to x86.

[dyung]

We run the debug info tests on GreenDragon on an Intel machine, so if they're not currently XFAIL'd they should be passing there (I'd double check but green.lab.llvm.org is having trouble currently). That said, if they're already disabled on arm I don't mind extending this to x86.

If they are passing, I would be curious what your configuration is, or if there is something I'm not setting up correctly. My setup is a just a clean OS install with XCode and some homebrew stuff installed so should have very little extra stuff, but there could be a configuration setting I might be missing?

[JDevlieghere]

GreenDragon is currently down and I have no ETA on when it'll be back up. In the meantime, I setup a new node somehow the tests are now failing there too. I suspect they might not be passing for good reasons on the existing Intel bot so let's disable them 👍

[dyung]

@adrian-prantl I noticed that you removed the XFAIL for all of darwin in 536e0eb, and then the test started to fail on one of my bots https://lab.llvm.org/buildbot/#/builders/280/builds/68, and I am guessing it will be failing on my other bot (https://lab.llvm.org/staging/#/builders/188) once it gets to your commit.

Are the bots I setup misconfigured in some manner? These tests have always been failing for me on the x86 bot at least (see https://lab.llvm.org/staging/#/builders/188/builds/400 for example), and it never failed previously on the AArch64 bot because it was previously XFAIL'd there as well.

[dyung]

@adrian-prantl I noticed that you removed the XFAIL for all of darwin in 536e0eb, and then the test started to fail on one of my bots https://lab.llvm.org/buildbot/#/builders/280/builds/68, and I am guessing it will be failing on my other bot (https://lab.llvm.org/staging/#/builders/188) once it gets to your commit.

Are the bots I setup misconfigured in some manner? These tests have always been failing for me on the x86 bot at least (see https://lab.llvm.org/staging/#/builders/188/builds/400 for example), and it never failed previously on the AArch64 bot because it was previously XFAIL'd there as well.

As expected, the two tests are failing now on my x86 bot as well: https://lab.llvm.org/staging/#/builders/188/builds/575

[JDevlieghere]

Can we just skip the test instead?

[dyung]

Can we just skip the test instead?

Do you mean by adding something like // REQUIRES: !system-darwin?

I would prefer to fix my bots so that they can correctly run the tests, but skipping the test is preferred to having them pass/fail on different configurations.

[adrian-prantl]

cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp and cross-project-tests/debuginfo-tests/llgdb-tests/static-member-2.cpp

These are not Dexter tests. They are end-to-end debug info tests. I also noticed that they were failing on Darwin with newer versions of LLDB, but I fixed the issues I found today. Are they still failing on your system?

[dyung]

cross-project-tests/debuginfo-tests/llgdb-tests/static-member.cpp and cross-project-tests/debuginfo-tests/llgdb-tests/static-member-2.cpp

These are not Dexter tests. They are end-to-end debug info tests. I also noticed that they were failing on Darwin with newer versions of LLDB, but I fixed the issues I found today. Are they still failing on your system?

Thanks for the fix! They are now passing on both of my bots (x86 and AArch64). Your updates exposed an issue with another test that we have seen before, and I've posted #85027 to XFAIL the test.