Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hello World app crashes on Arm32 Alpine 3.13 #47423

Closed
mthalman opened this issue Jan 25, 2021 · 20 comments
Closed

Hello World app crashes on Arm32 Alpine 3.13 #47423

mthalman opened this issue Jan 25, 2021 · 20 comments
Assignees
Labels
arch-arm32 area-Meta os-linux Linux OS (any supported distro)
Milestone

Comments

@mthalman
Copy link
Member

mthalman commented Jan 25, 2021

When attempting to run a .NET application on Arm32 Alpine 3.13, it crashes (core dump: core.zip).

This repros on both .NET runtime 5.0.2 and 6.0.0-alpha.1.21069.7.

This error does not happen when using the same architecture with Alpine 3.12. It also doesn't happen when using Arm64 Alpine 3.13.

Repro

Steps are to be run on an Arm32 device with Docker installed

  1. Install .NET 5.0 SDK
  2. mkdir app && cd app
  3. dotnet new console --no-restore
  4. Add a new file to the folder with the name Dockerfile and the following content:
FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim-arm32v7 as build

WORKDIR app
COPY *.csproj .
RUN dotnet restore

COPY . .
RUN dotnet build --no-restore


FROM build as publish
RUN dotnet publish --no-restore -c Release -o out


FROM mcr.microsoft.com/dotnet/nightly/runtime:5.0-alpine3.13-arm32v7
WORKDIR /app
COPY --from=publish /app/out ./
ENTRYPOINT ["dotnet", "app.dll"]
  1. sudo docker build -t test .
  2. sudo docker run --rm test

Expected: Outputs Hello World!
Actual: App crashes

Another result

I also found that adding code that makes a web request results in a NotTimeValid cert error with this exception stack:

Unhandled exception. System.AggregateException: One or more errors occurred. (The SSL connection could not be established, see inner exception.)
 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: NotTimeValid
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at app.Program.Main(String[] args) in /app/Program.cs:line 9

You can repro this by modifying the Program.cs file that gets generated in the above repro steps to include the following code:

static void Main(string[] args)
{
    var task = new System.Net.Http.HttpClient().GetAsync("https://www.microsoft.com");
    task.Wait();
    task.Result.EnsureSuccessStatusCode();
}

This repos in the same environment as described above and also works in the other environments that were mentioned (Alpine 3.12, Arm64 Alpine 3.13).

@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Net.Security untriaged New issue has not been triaged by the area owner labels Jan 25, 2021
@ghost
Copy link

ghost commented Jan 25, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

When attempting to run a .NET application on Arm32 Alpine 3.13, it crashes (core dump: core.zip).

This repros on both .NET runtime 5.0.2 and 6.0.0-alpha.1.21069.7.

This error does not happen when using the same architecture with Alpine 3.12. It also doesn't happen when using Arm64 Alpine 3.13.

Repro

Steps are to be run on an Arm32 device with Docker installed

  1. Install .NET 5.0 SDK
  2. mkdir app && cd app
  3. dotnet new console --no-restore
  4. Add a new file to the folder with the name Dockerfile and the following content:
FROM arm32v7/alpine:3.13 AS runtime

RUN apk add --no-cache \
        ca-certificates \
        \
        # .NET Core dependencies
        krb5-libs \
        libgcc \
        libintl \
        libssl1.1 \
        libstdc++ \
        zlib

ENV \
    # Configure web servers to bind to port 80 when present
    ASPNETCORE_URLS=http://+:80 \
    # Enable detection of running in a container
    DOTNET_RUNNING_IN_CONTAINER=true \
    # Set the invariant mode since icu_libs isn't included (see https://github.com/dotnet/announcements/issues/20)
    DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true

# Install .NET
ENV DOTNET_VERSION=5.0.2

RUN wget -O dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Runtime/$DOTNET_VERSION/dotnet-runtime-$DOTNET_VERSION-linux-musl-arm.tar.gz \
    && dotnet_sha512='e608a714767c8ddb6c397a0bd67e493c8546cd0d47c4c393f95d16a9e6aef0949c535a5ab8f0ccfe0642456f29c69d1b9a99561b37b9d24180bcea7e9417d5eb' \
    && echo "$dotnet_sha512  dotnet.tar.gz" | sha512sum -c - \
    && mkdir -p /usr/share/dotnet \
    && tar -C /usr/share/dotnet -oxzf dotnet.tar.gz \
    && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet \
    && rm dotnet.tar.gz


FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim-arm32v7 as build

WORKDIR app
COPY *.csproj .
RUN dotnet restore

COPY . .
RUN dotnet build --no-restore


FROM build as publish
RUN dotnet publish --no-restore -c Release -o out


FROM runtime
WORKDIR /app
COPY --from=publish /app/out ./
ENTRYPOINT ["dotnet", "app.dll"]
  1. sudo docker build -t test .
  2. sudo docker run --rm test

Expected: Outputs Hello World!
Actual: App crashes

Another result

I also found that adding code that makes a web request results in a NotTimeValid cert error with this exception stack:

Unhandled exception. System.AggregateException: One or more errors occurred. (The SSL connection could not be established, see inner exception.)
 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: NotTimeValid
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at app.Program.Main(String[] args) in /app/Program.cs:line 9

You can repro this by modifying the Program.cs file that gets generated in the above repro steps to include the following code:

static void Main(string[] args)
{
    var task = new System.Net.Http.HttpClient().GetAsync("https://www.microsoft.com");
    task.Wait();
    task.Result.EnsureSuccessStatusCode();
}

This repos in the same environment as described above and also works in the other environments that were mentioned (Alpine 3.12, Arm64 Alpine 3.13).

Author: mthalman
Assignees: -
Labels:

area-System.Net.Security, untriaged

Milestone: -

@MichaelSimons
Copy link
Member

This is blocking Add support for Alpine 3.13.

@janvorli
Copy link
Member

I can repro the problem, however the Alpine 3.13 docker image (even the original alpine:3.13) has apk broken, so I cannot install a debugger. Trying to install e.g. gdb:

apk add gdb
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/armv7/APKINDEX.tar.gz
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1913:
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.13/main: Permission denied
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.13/main: No such file or directory
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/armv7/APKINDEX.tar.gz
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:0D0D90AD:asn1 encoding routines:ASN1_TIME_adj:error getting time:crypto/asn1/a_time.c:330:
4150051728:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1913:
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.13/community: Permission denied
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.13/community: No such file or directory
ERROR: unable to select packages:
  gdb (no such package):
    required by: world[gdb]

I've found multiple reports of the same issue with Alpine 3.13 while people say that 3.12 works fine. I get the same behavior. One of such report is here: https://www.gitmemory.com/issue/alpinelinux/docker-alpine/135/761709893.

I wonder how we have succeeded creating the mcr.microsoft.com/dotnet/nightly/runtime:5.0-alpine3.13-arm32v7 image then. @mthalman any idea?

@mthalman
Copy link
Member Author

mthalman commented Jan 27, 2021

I wonder how we have succeeded creating the mcr.microsoft.com/dotnet/nightly/runtime:5.0-alpine3.13-arm32v7 image then. @mthalman Matt Thalman FTE any idea?

We don't require apk to produce the runtime and aspnet images because we don't install any additional packages. We just install the runtimes from .tar.gz files. However, we just started producing sdk images for 6.0 on arm32 Alpine 3.13 and we do use apk for that. Those builds are working fine. Here's build output:

Sending build context to Docker daemon  3.072kB

Step 1/5 : ARG REPO=mcr.microsoft.com/dotnet/aspnet
Step 2/5 : FROM $REPO:6.0-alpine3.13-arm32v7
 ---> 654e4c98a6fb
Step 3/5 : ENV     ASPNETCORE_URLS=     DOTNET_SDK_VERSION=6.0.100-preview.1.21075.9     DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false     DOTNET_USE_POLLING_FILE_WATCHER=true     NUGET_XMLDOC_MODE=skip     POWERSHELL_DISTRIBUTION_CHANNEL=PSDocker-DotnetSDK-Alpine-3.13-arm32
 ---> Running in df5bd9bf7101
Removing intermediate container df5bd9bf7101
 ---> 3b1029cb9a35
Step 4/5 : RUN apk add --no-cache         curl         icu-libs         git
 ---> Running in 97608eb04c86
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/armv7/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/armv7/APKINDEX.tar.gz
(1/8) Installing brotli-libs (1.0.9-r3)
(2/8) Installing nghttp2-libs (1.42.0-r1)
(3/8) Installing libcurl (7.74.0-r0)
(4/8) Installing curl (7.74.0-r0)
(5/8) Installing expat (2.2.10-r1)
(6/8) Installing pcre2 (10.36-r0)
(7/8) Installing git (2.30.0-r0)
(8/8) Installing icu-libs (67.1-r2)
Executing busybox-1.32.1-r0.trigger
OK: 46 MiB in 31 packages
Removing intermediate container 97608eb04c86
 ---> 6f0a820169ed
Step 5/5 : RUN wget -O dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Sdk/$DOTNET_SDK_VERSION/dotnet-sdk-$DOTNET_SDK_VERSION-linux-musl-arm.tar.gz     && dotnet_sha512='dc2ca32a68468f4a64adc276b1a08eb193e661c55d0aa9265c10772cc1b20ac28ae3154d5825459ac668d29a83841d1a22170f649bbee2acf9c3a3f1c069b024'     && echo "$dotnet_sha512  dotnet.tar.gz" | sha512sum -c -     && mkdir -p /usr/share/dotnet     && tar -C /usr/share/dotnet -oxzf dotnet.tar.gz ./packs ./sdk ./templates ./LICENSE.txt ./ThirdPartyNotices.txt     && rm dotnet.tar.gz     && dotnet help
 ---> Running in aa858add7f6d
�[91mConnecting to dotnetcli.azureedge.net (72.21.81.200:443)
�[0m�[91msaving to 'dotnet.tar.gz'
�[0m�[91mdotnet.tar.gz          0% |                                |     0 --:--:-- ETA
�[0m�[91mdotnet.tar.gz          7% |**                              |  9.8M  0:00:24 ETA
�[0m�[91mdotnet.tar.gz         18% |*****                           | 24.4M  0:00:13 ETA
�[0m�[91mdotnet.tar.gz         25% |********                        | 33.2M  0:00:11 ETA
�[0m�[91mdotnet.tar.gz         30% |*********                       | 39.8M  0:00:11 ETA
�[0m�[91mdotnet.tar.gz         33% |**********                      | 44.2M  0:00:11 ETA
�[0m�[91mdotnet.tar.gz         37% |***********                     | 49.0M  0:00:11 ETA
�[0m�[91mdotnet.tar.gz         46% |**************                  | 61.1M  0:00:09 ETA
�[0m�[91mdotnet.tar.gz         53% |*****************               | 70.7M  0:00:07 ETA
�[0m�[91mdotnet.tar.gz         60% |*******************             | 79.7M  0:00:06 ETA
�[0m�[91mdotnet.tar.gz         67% |*********************           | 89.3M  0:00:05 ETA
�[0m�[91mdotnet.tar.gz         70% |**********************          | 92.4M  0:00:05 ETA
�[0m�[91mdotnet.tar.gz         75% |************************        | 99.3M  0:00:04 ETA
�[0m�[91mdotnet.tar.gz         79% |*************************       |  104M  0:00:03 ETA
�[0m�[91mdotnet.tar.gz         82% |**************************      |  109M  0:00:03 ETA
�[0m�[91mdotnet.tar.gz         86% |***************************     |  113M  0:00:02 ETA
�[0m�[91mdotnet.tar.gz         86% |***************************     |  114M  0:00:02 ETA
�[0m�[91mdotnet.tar.gz         90% |*****************************   |  119M  0:00:01 ETA
�[0m�[91mdotnet.tar.gz         93% |*****************************   |  122M  0:00:01 ETA
�[0m�[91mdotnet.tar.gz         95% |******************************  |  126M  0:00:00 ETA
�[0m�[91mdotnet.tar.gz         99% |******************************* |  131M  0:00:00 ETA
�[0m�[91mdotnet.tar.gz        100% |********************************|  132M  0:00:00 ETA
�[0m�[91m'dotnet.tar.gz' saved
�[0mdotnet.tar.gz: OK

Welcome to .NET 6.0!
---------------------
SDK Version: 6.0.100-preview.1.21075.9

Telemetry
---------
The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
.NET SDK (6.0.100-preview.1.21075.9)
Usage: dotnet [runtime-options] [path-to-application] [arguments]

Execute a .NET application.

runtime-options:
  --additionalprobingpath <path>   Path containing probing policy and assemblies to probe for.
  --additional-deps <path>         Path to additional deps.json file.
  --depsfile                       Path to <application>.deps.json file.
  --fx-version <version>           Version of the installed Shared Framework to use to run the application.
  --roll-forward <setting>         Roll forward to framework version  (LatestPatch, Minor, LatestMinor, Major, LatestMajor, Disable).
  --runtimeconfig                  Path to <application>.runtimeconfig.json file.

path-to-application:
  The path to an application .dll file to execute.

Usage: dotnet [sdk-options] [command] [command-options] [arguments]

Execute a .NET SDK command.

sdk-options:
  -d|--diagnostics  Enable diagnostic output.
  -h|--help         Show command line help.
  --info            Display .NET information.
  --list-runtimes   Display the installed runtimes.
  --list-sdks       Display the installed SDKs.
  --version         Display .NET SDK version in use.

SDK commands:
  add               Add a package or reference to a .NET project.
  build             Build a .NET project.
  build-server      Interact with servers started by a build.
  clean             Clean build outputs of a .NET project.
  help              Show command line help.
  list              List project references of a .NET project.
  msbuild           Run Microsoft Build Engine (MSBuild) commands.
  new               Create a new .NET project or file.
  nuget             Provides additional NuGet commands.
  pack              Create a NuGet package.
  publish           Publish a .NET project for deployment.
  remove            Remove a package or reference from a .NET project.
  restore           Restore dependencies specified in a .NET project.
  run               Build and run a .NET project output.
  sln               Modify Visual Studio solution files.
  store             Store the specified assemblies in the runtime package store.
  test              Run unit tests using the test runner specified in a .NET project.
  tool              Install or manage tools that extend the .NET experience.
  vstest            Run Microsoft Test Engine (VSTest) commands.

Additional commands from bundled tools:
  dev-certs         Create and manage development certificates.
  fsi               Start F# Interactive / execute F# scripts.
  sql-cache         SQL Server cache command-line tools.
  user-secrets      Manage development user secrets.
  watch             Start a file watcher that runs a command when files change.

Run 'dotnet [command] --help' for more information on a command.
Removing intermediate container aa858add7f6d
 ---> 09db36ba3e54
Successfully built 09db36ba3e54
Successfully tagged dotnetdocker.azurecr.io/build-staging/966458/dotnet/nightly/sdk:6.0.100-preview.1-alpine3.13-arm32v7
Successfully tagged dotnetdocker.azurecr.io/build-staging/966458/dotnet/nightly/sdk:6.0-alpine3.13-arm32v7
Successfully tagged dotnetdocker.azurecr.io/build-staging/966458/dotnet/nightly/sdk:6.0-alpine-arm32v7
Successfully tagged dotnetdocker.azurecr.io/build-staging/966458/dotnet/nightly/sdk:6.0.100-preview.1-alpine3.13
Successfully tagged dotnetdocker.azurecr.io/build-staging/966458/dotnet/nightly/sdk:6.0-alpine3.13
Successfully tagged dotnetdocker.azurecr.io/build-staging/966458/dotnet/nightly/sdk:6.0-alpine

Correction:
We do use apk for the runtime-deps parent image that runtime is based on. That's obviously working as well.

@janvorli
Copy link
Member

Ok, it was caused by an outdated docker. I've reinstalled it from the docker's own repo and now I am able to install packages.

@janvorli
Copy link
Member

Interestingly enough, after I've upgraded docker, your sample above started to work for me - it prints Hello World now instead of crashing:

 sudo docker build -t test .
Sending build context to Docker daemon  4.096kB
Step 1/12 : FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim-arm32v7 as build
 ---> 476760c784eb
Step 2/12 : WORKDIR app
 ---> [Warning] The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in f2e694dab582
Removing intermediate container f2e694dab582
 ---> 375452349a22
Step 3/12 : COPY *.csproj .
 ---> afda8343ee64
Step 4/12 : RUN dotnet restore
 ---> [Warning] The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in be449f626ba9
  Determining projects to restore...
  Restored /app/47423.csproj (in 215 ms).
Removing intermediate container be449f626ba9
 ---> 7a2253427f6c
Step 5/12 : COPY . .
 ---> 9d87c4ba3670
Step 6/12 : RUN dotnet build --no-restore
 ---> [Warning] The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in 7d1cbd23afd5
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  47423 -> /app/bin/Debug/net5.0/47423.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.18
Removing intermediate container 7d1cbd23afd5
 ---> b232d21bd94a
Step 7/12 : FROM build as publish
 ---> b232d21bd94a
Step 8/12 : RUN dotnet publish --no-restore -c Release -o out
 ---> [Warning] The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in a91f588802cf
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  47423 -> /app/bin/Release/net5.0/47423.dll
  47423 -> /app/out/
Removing intermediate container a91f588802cf
 ---> de55f2b88cf3
Step 9/12 : FROM mcr.microsoft.com/dotnet/nightly/runtime:5.0-alpine3.13-arm32v7
 ---> 218bf2d2f738
Step 10/12 : WORKDIR /app
 ---> [Warning] The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in 9a7a14b00c5f
Removing intermediate container 9a7a14b00c5f
 ---> e1dfa5102da6
Step 11/12 : COPY --from=publish /app/out ./
 ---> b758a873d529
Step 12/12 : ENTRYPOINT ["dotnet", "47423.dll"]
 ---> [Warning] The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
 ---> Running in 4c51b38c75ac
Removing intermediate container 4c51b38c75ac
 ---> a6a7f1683f08
Successfully built a6a7f1683f08
Successfully tagged test:latest
janvorli@odroid:/mnt/ext/issues/47423$ sudo docker run --rm test
WARNING: The requested image's platform (linux/arm) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Hello World!

@janvorli
Copy link
Member

My version of docker is now

sudo docker --version
Docker version 20.10.2, build 2291f61

@karelz karelz added arch-arm32 os-linux Linux OS (any supported distro) labels Jan 28, 2021
@mthalman
Copy link
Member Author

@janvorli - I see you're running an arm64 machine. I'm using an arm32 machine (Raspberry Pi 4). I was using an older version of Docker (version 20.10.0). But even after upgrading it to 20.10.2, I still get the same crashing behavior.

@janvorli
Copy link
Member

Hmm, that's strange. I'll dig out my RPi 3 and see if I can repro it over there.

@janvorli
Copy link
Member

janvorli commented Feb 2, 2021

I have tried that on my Odroid XU4 (arm32 only device) with Ubuntu 20.04. The default docker reproed the issue, but after upgrading it to the one from the docker repo it worked fine (I had to use repo for bionic, as the one for focal didn't contain armhf packages for some reason). This time the docker version was 20.10.3.
I was googling a bit and found the alpine 3.13 release notes that describe a problem with time64 compatible system calls: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0#musl_1.2.
When I have tried to run ping to any address (I've tried a numeric one) from inside of the container before installing the new docker, I was getting the following error: ping: clock_gettime(MONOTONIC) failed. That seems to indicate that the problem is related to the release notes paragraph.
These notes also say that not only you have to have a docker >=19.03.9, but you also need to have libseccomp compatible with time64 installed on your host OS. You can check for that using scmp_sys_resolver -a arm clock_gettime64. If 403 is returned, time64 is supported. If -1 is returned, time64 is not supported.

@mthalman
Copy link
Member Author

mthalman commented Feb 3, 2021

Thanks for pointing out those release notes. A few key things here from the release notes:

Therefore, Alpine Linux 3.13.0 requires the host Docker to be version 19.03.9 (which contains backported moby commit 89fabf0) or greater and the host libseccomp to be version 2.4.2 (which contains backported libseccomp commit bf747eb) or greater.

For my RPi, Raspbian stable the latest version available for libseccomp is 2.3.3, less than the required 2.4.2.

Therefore, the following platforms are not suitable as Docker hosts for 32-bit Alpine Linux 3.13.0, due to containing out-of-date libseccomp: Amazon Linux 1 or 2, CentOS 7 or 8, Debian stable without debian-backports, Raspbian stable, Ubuntu 14.04 or earlier, and Windows.

Again, confirming the old version of libseccomp in Raspbian.

In order to run under old Docker or libseccomp versions... --security-opt=seccomp=unconfined can be passed with no default.json required

I tried executing my container with this option and it worked.

So I think all this confirms that these time64 requirements are the cause.

One open question remains, however: this issue also repros on our build machines (NVIDIA Jetson) that technically meet the requirements as defined in the release notes. I'll continue to investigate that.

@mthalman
Copy link
Member Author

mthalman commented Feb 3, 2021

Ok, it actually looks like I've been dealing with two separate issues here. The crash is definitely related to the time64 change in musl 1.2 By running on a compatible environment, that issue can be resolved except for the code sample that I provided above which makes a web request:

static void Main(string[] args)
{
    var task = new System.Net.Http.HttpClient().GetAsync("https://www.microsoft.com");
    task.Wait();
    task.Result.EnsureSuccessStatusCode();
}

@janvorli, have you tried that code in your environment? When I run that in a supported environment, I get the following exception:

Unhandled exception. System.AggregateException: One or more errors occurred. (The SSL connection could not be established, see inner exception.)
 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: NotTimeValid
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at app.Program.Main(String[] args) in /app/Program.cs:line 9

@janvorli
Copy link
Member

janvorli commented Feb 3, 2021

@mthalman I haven't tried this one, let me test it.

@janvorli
Copy link
Member

janvorli commented Feb 4, 2021

I am hitting the same issue. Since it is also related to time (based on the exception), I was wondering if we are actually hitting another thing that the release notes mention for 32 bit platforms:

All self-compiled packages must be manually rebuilt after upgrading, even if relocation/SONAME errors are not encountered.

To check if it is the issue, I've built .NET Core 5 using crossbuild and Alpine 3.13 rootfs (it required several little hacks to get there). I've built it from the same commit as the runtime that was in the docker container. Then I've copied the files into /usr/share/dotnet/shared/Microsoft.NETCore.App/5.0.2 and the issue is gone.
That means that we need to build runtime for ARM Alpine >=3.13 on Alpine 3.13. This doesn't affect the x64 / arm64 Alpine, as the breaking change is 32 bit OS only.

@karelz
Copy link
Member

karelz commented Feb 10, 2021

@janvorli what would be the right area path for the issue? PAL?

@janvorli
Copy link
Member

@karelz I would use meta, as the fix involves changes in rootfs build script in arcade, a patch to the pal.h header and change in the dotnet-buildtools-prereqs-docker repo.

@ghost ghost added this to Needs triage in Triage POD for Meta, Reflection, etc Feb 11, 2021
@joperezr joperezr removed the untriaged New issue has not been triaged by the area owner label Feb 11, 2021
@joperezr joperezr moved this from Needs triage to v-Next in Triage POD for Meta, Reflection, etc Feb 11, 2021
@joperezr joperezr added this to the 6.0.0 milestone Feb 11, 2021
@mthalman
Copy link
Member Author

What is the status of this? Is this something that can still get into Preview 2?

@mthalman
Copy link
Member Author

mthalman commented Jun 2, 2021

@janvorli - I'm still seeing the same failure as above on the upcoming 5.0.7 release. I believe this was the release you were expecting the fix to be in. Here's the output of dotnet --info in the environment which it fails in.

Host (useful for support):
  Version: 5.0.7
  Commit:  556582d964

.NET SDKs installed:
  No SDKs were found.

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

@janvorli
Copy link
Member

janvorli commented Jun 2, 2021

My local verification in Alpine 3.13 armv7 docker container running on arm64 Linux with the test from above have worked fine.

@janvorli
Copy link
Member

Everything is working fine now.

Triage POD for Meta, Reflection, etc automation moved this from Needs consultation to Done Jul 23, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Aug 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-arm32 area-Meta os-linux Linux OS (any supported distro)
Projects
No open projects
Development

No branches or pull requests

5 participants