[ARM32] `qemu: Unsupported syscall: 389` when running console app in Docker

[MichaelSimons]

While testing the new arm32v7 Docker images for .NET Core I'm seeing qemu: Unsupported syscall: 389 written to the output whenever I run a console application in an arm Docker container on a Windows 10 machine. I don't see this if I run the same Docker image on a physical arm32 device (e.g. raspberry pi).

Steps to reproduce

0. From a Windows 10 machine with Docker installed (Docker version 17.06.0-ce, build 02c1d87)
1. Create a new console app - dotnet new console
2. Add the following Dockerfile to your project

FROM microsoft/dotnet-nightly:2.0-sdk AS build-env
WORKDIR /app

# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore

# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out

# build runtime image
FROM msimons/dotnet-nightly:2-runtime-stretch-arm32v7
WORKDIR /app
COPY --from=build-env /app/out ./
ENTRYPOINT ["dotnet", "arm.dll"]

Note: The Dockerfile for msimons/dotnet-nightly:2-runtime-stretch-arm32v7 is contained in dotnet/dotnet-docker-nightly#328
3. Run the following from your project dir docker build -t armapp .
4. Run the image docker run --rm armapp

Expected Output of App

Hello World!

Output of App

qemu: Unsupported syscall: 389
Hello World!

Notes

1. The app's return code is 0
2. I tried running the image with the --privileged and --security-opt="seccomp=unconfined" options but see no difference in behavior.

[BruceForstall]

@dotnet/arm32-contrib @janvorli

[janvorli]

The syscall 389 on ARM32 seems to be the membarrier. According to the docs, it was added in Linux 4.11. dotnet core doesn't use it, so I assume one of our 3rd party dependencies try to call it to see if it is present. So I would assume the message is benign.

[mikedn]

Looks like this happens in CI too: https://ci.dot.net/job/dotnet_coreclr/job/master/job/arm_cross_debug_ubuntu16.04_prtest/4680/

FAILED   - JIT/Directed/coverage/importer/ldelemnullarr2/ldelemnullarr2.sh
               BEGIN EXECUTION
               /home/coreclr/bin/tests/Windows_NT.x64.Debug/Tests/coreoverlay/corerun ldelemnullarr2.exe
               qemu: Unsupported syscall: 389
               qemu: uncaught target signal 11 (Segmentation fault) - core dumped
               ./ldelemnullarr2.sh: line 243:   169 Segmentation fault      (core dumped) $_DebuggerFullPath "$CORE_ROOT/corerun" $ExePath $CLRTestExecutionArguments
               Expected: 100
               Actual: 139
               END EXECUTION - FAILED

[Tragetaschen]

membarrier has been in the ARM kernel since Linux 4.3: torvalds/linux@208473c

It looks like some tools like qemu (or in my case valgrind) don't yet have support for that.

[okodron]

Is it possible to use the latest version of ARM qemu with membarrier() support?

[davmason]

@RussKeldorph I am seeing this issue repeatedly in dotnet/coreclr#17142, it is blocking me from having a clean CI.

What is the guidance here? I have reset it 3 times and each time it takes hours to run and then fails. Should I start ignoring the armrel legs of the CI?

[RussKeldorph]

@davmason dotnet/coreclr#17028 is possibly more relevant. Yes, for now you can ignore these failures. I hope to have the job out of the default trigger soon since there is no outlook for a fix.

[BruceForstall]

Looks like dotnet/dotnet-docker-nightly#328 despite causing QEMU failures?

[MichaelSimons]

@janvorli - I am seeing a more severe variant of this with the new ARM SDK builds.

Steps to reproduce

1. Using Docker on a Windows 10 machine running in Linux container mode or a macOS.
2. Build the following Dockerfile - docker build -t dotnet-arm-sdk .

FROM arm32v7/buildpack-deps:stretch-scm

# Install .NET CLI dependencies
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        libc6 \
        libcurl3 \
        libgcc1 \
        libgssapi-krb5-2 \
        libicu57 \
        liblttng-ust0 \
        libssl1.0.2 \
        libstdc++6 \
        libunwind8 \
        zlib1g \
    && rm -rf /var/lib/apt/lists/*

# Install .NET Core SDK
ENV DOTNET_SDK_VERSION 2.1.300-preview2-008380

RUN curl -SL --output dotnet.tar.gz https://dotnetcli.blob.core.windows.net/dotnet/Sdk/$DOTNET_SDK_VERSION/dotnet-sdk-$DOTNET_SDK_VERSION-linux-arm.tar.gz \
    && dotnet_sha512='3f8276cd982f309236d5cac6656c9364cb190d1cb1ac0474acd67e16f35b8bb5d66fcb220342f0edb80c548a073d5e7f7330ceeeebdc88bceb8ea4836cd1abc6' \
    && echo "$dotnet_sha512 dotnet.tar.gz" | sha512sum -c - \
    && mkdir -p /usr/share/dotnet \
    && tar -zxf dotnet.tar.gz -C /usr/share/dotnet \
    && rm dotnet.tar.gz \
    && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet

# Configure Kestrel web server to bind to port 80 when present
ENV ASPNETCORE_URLS=http://+:80 \
    # Enable detection of running in a container
    DOTNET_RUNNING_IN_CONTAINER=true \
    # Enable correct mode for dotnet watch (only mode supported in a container)
    DOTNET_USE_POLLING_FILE_WATCHER=true \
    # Skip extraction of XML docs - generally not useful within an image/container - helps perfomance
    NUGET_XMLDOC_MODE=skip

# Trigger first run experience by running arbitrary cmd to populate local package cache
RUN dotnet help

Results:

...
Step 10/10 : RUN dotnet help
 ---> Running in ae71e039abc8
qemu: Unsupported syscall: 389

Welcome to .NET Core!
---------------------
Learn more about .NET Core: https://aka.ms/dotnet-docs
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs

Telemetry
---------
The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous and doesn't include command-line arguments. The data is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
qemu: Unsupported syscall: 345

ASP.NET Core
------------
Successfully installed the ASP.NET Core HTTPS Development Certificate.
To trust the certificate (Windows and macOS only) first install the dev-certs tool by running 'dotnet install tool dotnet-dev-certs -g --version 2.1.0-preview1-final' and then run 'dotnet-dev-certs https --trust'.
For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
The command '/bin/sh -c dotnet help' returned a non-zero code: 139

You can also reproduce this on Linux when using qemu but the setup is more complex.

[northtyphoon]

I hit the issue when trying to build an image based on microsoft/dotnet:2.1-runtime-stretch-slim-arm32v in Ubuntu 18.04 amd64 (qemu). Any suggestion?

https://github.com/Azure/dotnet-template-azure-iot-edge-module/blob/master/content/dotnet-template-azure-iot-edge-module/CSharp/Dockerfile.arm32v7

[Jacobh2]

Also seeing this issue on OSX 10.13.6 using image microsoft/dotnet:2.1-sdk-stretch-arm32v7 on step RUN dotnet restore -nowarn:msb3202,nu1503 -p:RestoreUseSkipNonexistentTargets=false. It gives the following output:

Step 9/18 : RUN dotnet restore -nowarn:msb3202,nu1503 -p:RestoreUseSkipNonexistentTargets=false
 ---> Running in 4cf6b676bcf0
qemu: Unsupported syscall: 389
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault

[eshvatskyi]

Same error :(

[janvorli]

@eshvatskyi which runtime version have you tried it with? We have fixed a bunch of ARM issues in 2.1.5.

[eshvatskyi]

@eshvatskyi which runtime version have you tried it with? We have fixed a bunch of ARM issues in 2.1.5.

FROM microsoft/dotnet:2.1-sdk-stretch-arm32v7
...
RUN dotnet restore -r linux-arm