Skip to content

[ppc64le] Source build crashing for mono runtime. #85089

Open
Open
[ppc64le] Source build crashing for mono runtime.#85089
Labels
area-Infrastructure-mono
Milestone
Future
@vikasgupta8

Description

@vikasgupta8
vikasgupta8
opened on Apr 20, 2023

Description

Observed crash while buildig source build on RHEL9.

The crash has happend in function "mono_object_handle_isinst".

https://github.com/dotnet/runtime/blob/v7.0.4/src/mono/mono/metadata/object.c#L6621

Looking at the below stacktrace we could see the klass variable is NULL and has invalid value.


Thread 20 (Thread 0x20002d35f080 (LWP 2829778) ".NET ThreadPool"): [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#0  0x00002000006142b4 in wait4 () from /lib64/libc.so.6 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#1  0x000020000061411c in waitpid () from /lib64/libc.so.6 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#2  0x0000200000c35650 in dump_native_stacktrace (signal=<optimized out>, mctx=<optimized out>) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/mini/mini-posix.c:843 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#3  mono_dump_native_crash_info (signal=<optimized out>, mctx=0x20002d359558, info=<optimized out>) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/mini/mini-posix.c:870 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#4  0x0000200000be4bac in mono_handle_native_crash (signal=0x2000008ebb91 "SIGSEGV", mctx=0x20002d359558, info=0x20002d35a5a0) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/mini/mini-exceptions.c:3005 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#5  0x0000200000b33ac8 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x20002d35a5a0, context=0x20002d359820, debug_fault_addr=0x20) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/mini/mini-runtime.c:3774 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#6  <signal handler called> [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#7  mono_object_handle_isinst (obj=..., klass=0x0, error=0x20002d35a8f8) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/object.c:6621 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#8  0x0000200000a229f4 in mono_object_isinst_checked (obj_raw=0x20026ddb85b0, klass=0x0, error=0x20002d35a8f8) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/object.c:6604 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#9  0x00002000009e359c in mono_marshal_isinst_with_cache (obj=0x20026ddb85b0, klass=<optimized out>, cache=0x20027022f458) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/marshal.c:4280 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#10 0x000020000134bc70 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#11 0x000020000138f244 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#12 0x000020027c6965c0 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#13 0x000020026e169d90 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#14 0x000020026e168ba4 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#15 0x00002002844acfb8 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#16 0x00002002844ac4f4 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#17 0x00002002844ab110 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#18 0x00002002844a87e4 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#19 0x00002002844a56cc in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#20 0x00002002844a3910 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#21 0x00002002844a0778 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#22 0x000020028449f678 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#23 0x00002002844708a4 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#24 0x000020028446dea4 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#25 0x000020027e714178 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#26 0x000020027e70838c in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#27 0x000020027e6f1ac0 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#28 0x000020027e6eea68 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#29 0x000020027e6ee6d8 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#30 0x000020027e6eda70 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#31 0x000020027e6eb778 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#32 0x000020027e6eaa60 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#33 0x000020027e6e901c in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#34 0x000020027e6e5318 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#35 0x000020027e6e38e8 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#36 0x000020027e6e0cf0 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#37 0x000020027e6dd080 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#38 0x000020027e6dcb70 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#39 0x000020027e6db9b8 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#40 0x000020027e6db688 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#41 0x000020027e6db470 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#42 0x000020026f0e906c in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#43 0x000020000eb945f8 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#44 0x000020000ebac194 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#45 0x000020000eb93930 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#46 0x000020000eb93260 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#47 0x000020000ebabf70 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#48 0x000020000eba3d74 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#49 0x000020000eba00a8 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#50 0x000020000eb9cdd4 in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#51 0x000020000eb86acc in ?? () [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#52 0x0000200000b3725c in mono_jit_runtime_invoke (method=0x0, obj=0x2000015979a8, params=0x20026ddb85b0, exc=<optimized out>, error=0x20027022f458) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/mini/mini-runtime.c:3571 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#53 0x0000200000a129c8 in do_runtime_invoke (method=0x10febd248, obj=<optimized out>, params=0x0, exc=0x0, error=0x20002d35e518) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/object.c:2415 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#54 mono_runtime_invoke_checked (method=0x10febd248, obj=<optimized out>, params=0x0, error=0x20002d35e518) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/object.c:2583 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#55 0x0000200000a38fe8 in start_wrapper_internal (start_info=0x0, stack_ptr=<optimized out>) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/threads.c:1213 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#56 0x0000200000a38d00 in start_wrapper (data=0x20004c002ef0) at /root/7.0.2/dotnet7.0/dotnet-v7.0.104/src/runtime/artifacts/source-build/self/src/src/mono/mono/metadata/threads.c:1264 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
#57 0x00002000005b0534 in start_thread () from /lib64/libc.so.6 [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]
Backtrace stopped: previous frame inner to this frame (corrupt stack?) [/root/dotnet-v7.0.103/src/sdk/artifacts/source-build/self/src/src/Tasks/Microsoft.NET.Build.Tasks/Microsoft.NET.Build.Tasks.csproj::TargetFramework=net7.0]

Reproduction Steps

Steps to reproduce:
1.      Install .Net7 SDK with following command
$ sudo dnf install -y dotnet-sdk-7.0{,-source-built-artifacts}

2.      Install dependencies

$ sudo dnf group install -y "Development Tools" --skip-broken
$ sudo dnf -y install \
         clang \
         cmake \
         findutils \
         git \
         glibc-langpack-en \
         hostname \
         krb5-devel \
         libicu-devel \
         llvm \
         lttng-ust-devel \
         make \
         openssl-devel \
         python3 \
         zlib-devel

4.      On a x86_64 machine create a .NET7.0.104 source tarball

$ git clone https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Finstaller.git&data=05%7C01%7Cvlbrez%40microsoft.com%7C62006a1f44014b26804b08db3509f5d2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638162089414042687%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MLnxqs94wszFb0RQXUqmVXjQouzyZIo9TQwatMgVq3E%3D&reserved=0
$ git checkout v7.0.104
$ ./build.sh /p:ArcadeBuildTarball=true

5.      Copy the tarball generated on x86  over to $HOME/dotnet-sdk-source-7.0.104.tar.gz on the ppc64le machine.
6.      Then run the following script

#!/bin/bash
    set -ueo pipefail
    set -x

    export OPENSSL_ENABLE_SHA1_SIGNATURES=1

    while true; do
        rm -rf dotnet-v7.0.104
        mkdir -p dotnet-v7.0.104
        pushd dotnet-v7.0.104
        tar xf ~/dotnet-sdk-source-7.0.104.tar.gz

        cp -a /usr/lib64/dotnet previous-sdk
        mkdir previously-source-built-artifacts
        pushd previously-source-built-artifacts
       tar xf /usr/lib64/dotnet/source-built-artifacts/Private.SourceBuilt.Artifacts.7.0.104.tar.gz
        popd
        ./build.sh \
           --with-sdk previous-sdk \
           --with-packages previously-source-built-artifacts \
           --use-mono-runtime
        popd
    done

Expected behavior

There should not be crash.

Actual behavior

Source build is crashing.

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-Infrastructure-mono

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions