Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Alpine support #192

Merged
merged 2 commits into from
Apr 25, 2022
Merged

Conversation

ayakael
Copy link
Contributor

@ayakael ayakael commented Mar 16, 2022

Description

Alpine support

Modifications

  • replaced all non-portable shebangs with #!/usr/bin/env bash
  • assemblies-valid: explicitely call bash for built-in command command when looking for dotnet binary
  • cgroup-limit: disable when init system not systemd
  • distribution-packages: disabled for Alpine as test uses rpm
  • iblttng-ust_sys-sdt.h: adapted for use with apk
  • lttng: look for babeltrace2 if cannot find babeltrace
  • man-page: adapted for use with apk
  • omnisharp: disabled for Alpine as omnisharp has no linux-musl release
  • runtime-id: adapted to parse Alpine runtime-id correctly

Problematic tests

Related PRs / Issues

dotnet/installer#13074

Made as part of Alpine Linux dotnet6 packaging project, see dotnet/source-build#2782

@ayakael
Copy link
Contributor Author

ayakael commented Mar 17, 2022

Bug with publish-ready-to-run on aarch64 (and presumably armv7) platform, logs:

$./test.sh 6.0.3
+ dotnet new console --no-restore
The template "Console App" was created successfully.
++ ../runtime-id
+ dotnet publish -r alpine.3.16-arm64 /p:PublishReadyToRun=true
Microsoft (R) Build Engine version 17.0.0+c9eb9dd64 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj (in 37.6 sec).
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(1110,5): warning NETSDK1179: One of '--self-contained' or '--no-self-contained' options are required when '--runtime' is used. [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
  publish-ready-to-run -> /home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/bin/Debug/net6.0/alpine.3.16-arm64/publish-ready-to-run.dll
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003: The specified task executable "/home/build/.nuget/packages/microsoft.netcore.app.crossgen2.linux-musl-arm64/6.0.3/tools/crossgen2" could not be run. System.ComponentModel.Win32Exception (2): An error occurred trying to start process '/home/build/.nuget/packages/microsoft.netcore.app.crossgen2.linux-musl-arm64/6.0.3/tools/crossgen2' with working directory '/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run'. No such file or directory [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003:    at System.Diagnostics.Process.Start() [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003:    at Microsoft.NET.Build.Tasks.RunReadyToRunCompiler.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(463,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]
/usr/share/dotnet/sdk/6.0.103/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.CrossGen.targets(351,5): error NETSDK1096: Optimizing assemblies for performance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false. [/home/build/alpine-test-suite/testing/dotnet6-build/src/dotnet-regular-tests-6207e217336d4f9c92add85b310c8a4b1ac0e6b1/publish-ready-to-run/publish-ready-to-run.csproj]

Caused by crossgen2 pointing to non-musl libc:

$ldd /home/build/.nuget/packages/microsoft.netcore.app.crossgen2.linux-musl-arm64/6.0.3/tools/crossgen2
	/lib/ld-linux-aarch64.so.1 (0xffff8fce9000)
	libpthread.so.0 => /lib/ld-linux-aarch64.so.1 (0xffff8fce9000)
	libdl.so.2 => /lib/ld-linux-aarch64.so.1 (0xffff8fce9000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xffff8fac8000)
	libm.so.6 => /lib/ld-linux-aarch64.so.1 (0xffff8fce9000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xffff8faa3000)
	libc.so.6 => /lib/ld-linux-aarch64.so.1 (0xffff8fce9000)
Error loading shared library ld-linux-aarch64.so.1: No such file or directory (needed by /home/build/.nuget/packages/microsoft.netcore.app.crossgen2.linux-musl-arm64/6.0.3/tools/crossgen2)

This issue does not exist on x64. Does this mean that the Crossgen2 nuget package for musl-arm64 on nuget.org is broken?

@ayakael
Copy link
Contributor Author

ayakael commented Mar 17, 2022

Indeed, this is a bug with the official build of CrossGen2 for musl-arm64 platform, as the one built by Alpine links to the correct libc. Where should I report this bug?

@omajid
Copy link
Member

omajid commented Mar 17, 2022

Where should I report this bug?

Thanks for finding this. Crossgen2 is part of dotnet/runtime. Please report it there: https://github.com/dotnet/runtime/issues/new?assignees=&labels=&template=01_bug_report.yml

@ayakael
Copy link
Contributor Author

ayakael commented Mar 17, 2022

Where should I report this bug?

Thanks for finding this. Crossgen2 is part of dotnet/runtime. Please report it there: https://github.com/dotnet/runtime/issues/new?assignees=&labels=&template=01_bug_report.yml

Thanks! I'll do so. I suspect the same for arm:

    /builds/ayakael/aports/testing/dotnet6-build/src/source-build-tarball-6.0.103/src/aspnetcore.c911002ab43b7b989ed67090f2a48d9073d5118d/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj(446,5): error MSB3073: The command ""/builds/ayakael/aports/testing/dotnet6-build/src/source-build-tarball-6.0.103/src/aspnetcore.c911002ab43b7b989ed67090f2a48d9073d5118d/artifacts/source-build/self/package-cache/microsoft.netcore.app.crossgen2.linux-musl-arm/6.0.3/tools/crossgen2" --targetarch:arm --targetos:linux -O @"/builds/ayakael/aports/testing/dotnet6-build/src/source-build-tarball-6.0.103/src/aspnetcore.c911002ab43b7b989ed67090f2a48d9073d5118d/artifacts/source-build/self/src/artifacts/obj/Microsoft.AspNetCore.App.Runtime/Release/net6.0/linux-musl-arm/crossgen/PlatformAssembliesPathsCrossgen2.rsp" --perfmap --perfmap-format-version:1 --perfmap-path:"/builds/ayakael/aports/testing/dotnet6-build/src/source-build-tarball-6.0.103/src/aspnetcore.c911002ab43b7b989ed67090f2a48d9073d5118d/artifacts/source-build/self/src/artifacts/bin/Microsoft.AspNetCore.App.Runtime/Release/net6.0/linux-musl-arm/" -o:"/builds/ayakael/aports/testing/dotnet6-build/src/source-build-tarball-6.0.103/src/aspnetcore.c911002ab43b7b989ed67090f2a48d9073d5118d/artifacts/source-build/self/src/artifacts/bin/Microsoft.AspNetCore.App.Runtime/Release/net6.0/linux-musl-arm/Microsoft.Extensions.Caching.Abstractions.dll" "/builds/ayakael/aports/testing/dotnet6-build/src/source-build-tarball-6.0.103/src/aspnetcore.c911002ab43b7b989ed67090f2a48d9073d5118d/artifacts/source-build/self/package-cache/microsoft.extensions.caching.abstractions/6.0.0/lib/netstandard2.0/Microsoft.Extensions.Caching.Abstractions.dll"" exited with code 139.    

Although I can't actually ldd it, as I don't have shell access to an arm environment.

@omajid
Copy link
Member

omajid commented Mar 17, 2022

I am not sure about the arm32 error.

The exit code, 139, is computed by adding 128 to the signal code. 139 - 128 is 11, which is the signal value for SIGSEGV.

The crossgen of Microsoft.Extensions.Caching.Abstractions.dll during the aspnetcore build is also the first time the just-built crossgen2 (and the rest of the coreclr runtime) is executed. Every time I have run into this in the past, this has been as symptom that the underlying runtime was broken on the platform.

@@ -8,6 +8,9 @@ runtime_id=$(../runtime-id)
# This might be the final/only netstandard version from now on
netstandard_version=2.1

# disabled for alpine
[ -z "${runtime_id##alpine*}" ] && { echo Disabled for Alpine; exit 0; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally add the RID to ignoredRIDs element in the test.json file. That said, I am not sure our test runner knows about linux-musl- yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what I figured, which is what prompted this. Eventually we should test using apk, like was done with other tests (like man-page, but alas I don't have the time for that, and I'd need to brush up on python.

@ayakael
Copy link
Contributor Author

ayakael commented Mar 19, 2022

PR as it is works on Alpine pipelines, as long as the following is disabled:
publish-ready-to-run / publish-ready-to-run-linux (on arm / arm64 only, fixed by dotnet/runtime#66814)
omnisharp (no linux-musl release)
liblttng-ust_sys-sdt.h / lttng (incompatibility with latest lttng - 2.13)

@ayakael
Copy link
Contributor Author

ayakael commented Mar 19, 2022

I am not sure about the arm32 error.

The exit code, 139, is computed by adding 128 to the signal code. 139 - 128 is 11, which is the signal value for SIGSEGV.

The crossgen of Microsoft.Extensions.Caching.Abstractions.dll during the aspnetcore build is also the first time the just-built crossgen2 (and the rest of the coreclr runtime) is executed. Every time I have run into this in the past, this has been as symptom that the underlying runtime was broken on the platform.

Strangest thing is that rhis doesn't always occur. Build of armv7, in my experience, has been unstable: random SIGSEGV that occur in one build, but not another. Since I don't have shell access to any armv7 platforms, I havn't been able to debug it.

@ayakael
Copy link
Contributor Author

ayakael commented Apr 3, 2022

Testing suite was successfully integrated in the Alpine package builder, using this patch and by disabling the following tests:

man-pages: Alpine package builder runs check function before packaging. man-pages tests against build packages, thus is disabled in the context of the build script. I have tested this to work with no issue.
distribution-package: Requires more work to adapt to apk package manager
bash-completion: Alpine package builder runs check function before packaging. bash-completion expects completion to be in installed location. There is no doubt a way to test a non-installed bash completion script, but alas havn't done this yet. This has been tested to work when manipulating packages.
release-version-sane: This fails if release is not latest version, so I've disabled it in the context of building as it isn't useful for that use-case.
omnisharp: No release for linux-musl, request made OmniSharp/omnisharp-roslyn#2366
createdump-aspnet: Permission issue in lxc / pipeline environment
workload: Permission issue in lxc / pipeline environment
liblttng-ust_sys-sdt.h: no 'NT_STAPSDT' on Alpine's lttng-ust package
lttng: broken starting with lttng-ust 2.13, see dotnet/runtime#57784
{bundled,system}-libunwind: Ideally, bundled should always be disabled, but as system libunwind is broken on aarch64/armv7, it is the system test that is disabled on those platforms
publish-ready-to-run: broken due to crossgen2 nupkg on musl-arm64 / musl-arm platform being broken. Should be fixed with dotnet/runtime#66814, thus on runtime 6.0.4

@ayakael
Copy link
Contributor Author

ayakael commented Apr 9, 2022

@omajid test-template sometimes fails due to timeout. Any way to make the timeout longer?
edit Fixed! Found the --timeout argument.

@omajid
Copy link
Member

omajid commented Apr 12, 2022

Hey! Sorry about sitting on this without any movement.

There's a few things here that I think we should probably fix in https://github.com/redhat-developer/dotnet-bunny first and then merge this. Would be okay with waiting, or would you rather we got this in and then fix https://github.com/redhat-developer/dotnet-bunny later (and undo some changes being made here)?

1 similar comment
@omajid

This comment was marked as duplicate.

@ayakael
Copy link
Contributor Author

ayakael commented Apr 12, 2022

Hey! Sorry about sitting on this without any movement.

There's a few things here that I think we should probably fix in https://github.com/redhat-developer/dotnet-bunny first and then merge this. Would be okay with waiting, or would you rather we got this in and then fix https://github.com/redhat-developer/dotnet-bunny later (and undo some changes being made here)?

No worries, ya'll can take your time on this, as I've already integrated this patch with Alpine's patching logics.

omajid added a commit to omajid/dotnet-bunny that referenced this pull request Apr 19, 2022
Parse `ldd --version` output and use that to detect musl-based
platforms. Add the additional RIDs `linux-musl` and `linux-musl-$ARCH`
for them, so those strings can be used to filter tests on those
platforms.

Also remove the last version number string from Alpine, Rocky, Alma
and Oracle Linux.

Add alpine to CI matrix, and enable unit tests for it. I am not enabling
reproducers on it right now, because that needs fixes from
redhat-developer/dotnet-regular-tests#192 to be
merged first.
omajid added a commit to redhat-developer/dotnet-bunny that referenced this pull request Apr 19, 2022
Parse `ldd --version` output and use that to detect musl-based
platforms. Add the additional RIDs `linux-musl` and `linux-musl-$ARCH`
for them, so those strings can be used to filter tests on those
platforms.

Also remove the last version number string from Alpine, Rocky, Alma
and Oracle Linux.

Add alpine to CI matrix, and enable unit tests for it. I am not enabling
reproducers on it right now, because that needs fixes from
redhat-developer/dotnet-regular-tests#192 to be
merged first.
@omajid
Copy link
Member

omajid commented Apr 20, 2022

I made some improvements via 9c1fa19 that I would like to add on top of this PR. It requires redhat-developer/dotnet-bunny@71880bd to work, though.

@ayakael any chance you could try this out? If it looks okay to you, I would like to merge this PR and then add my commit on top.

@ayakael
Copy link
Contributor Author

ayakael commented Apr 25, 2022

I made some improvements via 9c1fa19 that I would like to add on top of this PR. It requires redhat-developer/dotnet-bunny@71880bd to work, though.

@ayakael any chance you could try this out? If it looks okay to you, I would like to merge this PR and then add my commit on top.

On it, I'll come back to you shortly.

@ayakael
Copy link
Contributor Author

ayakael commented Apr 25, 2022

Looks good on my side!

- Use recent features from dotnet-bunny to filter on linux-musl

- Fix some weird looking process launching code in assemblies-valid

- Handle more musl-based distributions in runtime-id script
@omajid omajid merged commit ddc8576 into redhat-developer:main Apr 25, 2022
@ayakael ayakael deleted the alpine-support branch June 15, 2022 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants