Enabling Linux ARM32 for .NET Core #2548
Comments
|
As I have mentioned in the discussion in dotnet/core-setup#724, I believe that we should be consistent in the way we do cross build over our repos. That means to use rootfs and to do the cross build using clang and not GCC. The same way we do it in CoreCLR and CoreFX. |
|
@gkhanna79 AFAIK default rid is ubuntu.14.04-arm for coreclr ARM and default ARM rootfs is trusty which is ubuntu 14.04. |
|
@hqueue @gkhanna79 RaspberryPI2 used ubuntu 14.04. (but no longer maintained) |
|
@hseok-oh If Ubuntu 14.04 is not maintained and supported, is there a specific reason to support it for Linux Arm32? If not, it will simplify bunch of things (e.g. libcoreclrtraceptprovider.so builds just fine for 16.04 but needs additional work for 14.04). What do you think? |
I don't know who chose those defaults or why. Could it be cause Tizen OS is based upon Ubuntu 14.04? do we build libcoreclrtraceptprovider.so for Tizen? |
|
@kouvel This is the Linux Arm32 main work-list issue. |
|
CC @schellap |
@gkhanna79 |
gkhanna79
changed the title
|
Thanks for the explanation @hqueue. For now, we can continue to build for both 14.04 and 16.04 and see how far we will need the 14.04 implementation. With regards to the emulator, I noticed that in CoreCLR repo, we bootup the emulator and setup the rootfs within it and then perform the build. This is different from the build instructions we have at https://github.com/dotnet/coreclr/blob/master/Documentation/building/cross-building.md where it comprises of two steps:
We strive hard to ensure that CI builds the same way as any developer would. Since the above steps are how any developer in the community would be expected to build for Arm32, what do you think of updating the CI script to do just that and not rely on emulator to be present for performing the build? Secondly, the emulator should only be required to deploy binaries and execute tests. I notice that we only run 22 tests within the emulator, even though the repo has 11K+ tests. Do these 22 tests add any value from the CI perspective (e.g. here is a log of a recent PR that shows the 22 tests that ran - https://ci.dot.net/job/dotnet_coreclr/job/master/job/arm_emulator_cross_debug_ubuntu_prtest/1080/consoleFull) since they dont seem to cover key scenarios where breaks maybe introduced? |
|
@gkhanna79 For the first question, I will call @sjsinju @wateret, since I'm not aware of the latest of ARM CI. @sjsinju @wateret Can you please answer the first question above regarding CI and ARM emulator ? For the second one, I also think tests are not sufficient to cover key sccenarios. We will look into them. |
|
@gkhanna79 One more thing that may be confusing is that the CI has arm-softfp rootfs only which cannot be created by 'any developers'. There is my work on CoreFX for enabling arm-hardfp(ubuntu 14.04). After it's done I will apply it to CoreCLR as well. |
|
@gkhanna79 I couldn't find where libuv.so is built. Could you give me an advice? |
|
@jyoungyun not sure if it's the way libuv is built for .NET, but for raspberry pi the instructions so far have been to download it from the libuv project & compile it: See the these comments from @leemgs related to dotnet/coreclr#6321 That worked for me to get Kestrel up & running on Raspberry Pi; but note that newer versions of libuv are now available at http://dist.libuv.org/dist/ |
@wateret My point is that we should decouple cross building in CI from emulator. The nuance is that in CI, we setup rootfs from emulator and then perform cross-build while a regular developer will not. My suggestion is to fix the script to do the following:
|
|
@hqueue did you find anything interesting about running the 22 tests in CI? |
|
@qmfrederik Thank you for your answer. What you are talking about is how to download libuv.so directly. But I want to know where the libuv*.nupkg is generated. The NETCoreApp is dependent on libuv module so I wanna generate libuv*.nupkg for arm. |
Just make things clear, the emulator image is nothing but pre-built rootfs. Of course the emulator also needs it for running tests. For the first one you suggested, basically I agree with you. However it takes pretty long to build rootfs with |
@gkhanna79 Those 22 tests are TCs where we observed regression frequently when brining-up CoreCLR for Linux ARM32. Therefore the coverage of 22 tests does not cover the code which is not modified freqeuently. However we cannot run all TCs (11K+) with ARM emulator becasue it took very long time and we faced timeout in CI environment. I also think we need more TCs for CI, but not sure about how to choose candidates. Can you suggest any options ? cc: @jyoungyun |
|
@jyoungyun I think the libuv NuGet package is located here: https://github.com/aspnet/libuv-package?files=1 |
|
@qmfrederik Thank you! I will check that repository. Finally, I succeeded in obtaining a dotnet-ubuntu-arm.1.2.0-beta-001206-00.tar.gz file, which confirmed that it works well on Raspberry PI2. I built the core-setup repository using the local nupkg source and these are the results built from CoreCLR and CoreFX(including dotnet/corefx#14655 patch). If CoreCLR and CoreFX nupkgs are uploaded in the NugetServer or anywhere, we can generate the .NET Core Runtime for Ubuntu ARM. I have tested only on Ubuntu 14.04 and will be testing on 16.04 later. |
|
Awesome! If there's any way for you to share the NuGet packages/tarballs, I'd be happy to try them out on a RPi 2 & 3, and see how far I get with our .NET Core-based product :) |
This is excellent news @jyoungyun! Thank you all for driving this through :) The LibUV package does come from ASP.NET repository. @Eilon Can you please advise which repo to enlist to build this package from? |
This is the work I am tracking to get done as part of the E2E pipeline build support across the three .NET Core repos (CoreCLR, CoreFX, Core-Setup). |
That is interesting. For me, creating a new rootfs takes about 7 or so mins on my USB 3.0 SSD for both Trusty and Xenial builds. Everything else is cost of a regular cross build from there on. Hence, my question around why we need to pickup rootfs from emulator and not just fetch it everytime. Building upon this, to enable CI build for Ubuntu 16.04 arm, are you planning to add another emulator that will contain rootfs?
Do you have any automation around tests on real HW? My current thinking is to evaluate doing so in favor of emulator int he foreseeable future. |
|
I can also confirm that creating a new rootfs takes a couple of minutes on my Linux box. It is probably related to the speed of the internet connection, since the process fetches all the necessary packages. |
|
The libuv source code is 3rd party and is here: https://github.com/libuv/libuv The ASP.NET team's build process for compiling libuv is in our own repo here: https://github.com/aspnet/libuv-build/ And it references the libuv source code via a Git submodule: https://github.com/aspnet/libuv-build/tree/dev/submodules And finally, the ASP.NET team has a repo for taking the libuv binaries and packaging them into a NuGet package: https://github.com/aspnet/libuv-package |
|
I don't think it works on Pi Zero. I don't think anyone has looked at why. |
|
@olegsavelos the debian.8-armel build will not work on a raspberry pi zero. All of the builds here are for armv7 devices, with the different versions selecting between hard float ABI and soft float ABI. The Pi Zero (and the original Pi) is an armv6 device. I do not know what it will take to get armv6 working and building, but it could range from fairly easy (changing compiler flags) to very difficult. I just don't know. |
|
Can anyone shed some light on Armv6 support ? |
|
@jyoungyun , will the .net core tizen build work on an image including https://github.com/TizenTeam/meta-tizen on yocto? |
|
@olegsavelos If you meant PI Zero with Broadcom BCM2835, I'm afraid it's not easy to work dotnet-debian.8-armel.1.2.0-beta-001271-00.tar.gz on your Pi Zero. I also agree with @ThadHouse that PI Zero is based on armv6 and it could be range from easy to very difficult. AFAIK current dotnet for ARM make use of Thumb-2 ISA which is not available in armv6 and vfp feature which is mostly available in armv7, but only optinally available in armv6. Armv6 support old Thumb not Thumb2, therefore you may have to build (1) all native component of dotnet for armv6 and also (2) may have to update JIT compiler and sub-component(written in assmebly language) of CoreCLR and CoreFX to make use of Thumb ISA and supported vfp feature of Broadcom BCM2835 instead of armv7. But I can't estimate how much effeort will be needed. |
|
Mono seems to be working fine on armv6 so wouldn't it be possible to port the portions of code related to armv6 to .net core? In any case i think PI Zero is excellent platform for wide range of solutions and it would be shame not to support it. |
|
Can anyone tell me when will I be able to build and run programs on arm32 linux? thanks |
|
@rahulreddy65 If you are using Ubuntu 14.04 or Ubuntu 16.04 for arm, I think you can run C# programs on arm32 linux by installing binary from https://github.com/dotnet/core-setup#daily-builds.
If you want to build C# program from arm32 linux, it's not available yet. However you can build C# program from other environment, e.g. x64 linux or x64 Windows, and run the program on Ubuntu arm. And there is another way you can just build and run C# program for arm32 linux with runtime framework version 2.0.0-beta-001620-00 and later. Please take a look at https://github.com/dotnet/core/blob/master/samples/RaspberryPiInstructions.md |
|
@olegsavelos I think you'd better open an issue about supporting armv6 at https://github.com/dotnet/coreclr, because most of required works are relevant to coreclr and experts who can answer your questions(e.g. supporting new CPU) are there too :) |
|
@hqueue Thanks ! Will do that :) |
|
Finally, I got below results!!! The performance is very.... low, but Raspberry PI can build cs file itself! |
|
@jyoungyun are these results from release build of coreclr? |
|
@janvorli All of the nupkg used in the cli build got downloaded from myget server so this runtime(coreclr) is release build with -O1 option. If |
|
@jyoungyun it was discovered in another issue that dotnet run command adds quite a lot of overhead to the start of the managed app (on x64 Linux, it adds about 2 seconds) compared to passing the managed assembly directly to dotnet without the "run". |
|
@janvorli In raspberry PI3, passing dll as a directy argument is about 40 seconds faster than using dotnet run. This seems to be quite a lot of overhead in dotnet run command... |
|
@jyoungyun Thank you for measuring it! We really need to do something with it. I'll add this information to the other issue where the slowness was reported for x64. |
|
@janvorli @jyoungyun I think one of huge timz difference can be due to lack of native image for arm cli, which may be available for x64 cli itself. |
|
Well, even on x64, there is a huge difference in those two ways of launching the app. See https://github.com/dotnet/cli/issues/6241 |
|
I see a lot of progress related to the Raspberry PI 2/3 and Ubuntu... but what is required to get my dotnet core 2 project running on an i.MX7 (Cortex-A7) with an angstrom based linux? From my limited understanding the architecture should be armhf, which is the default architecture in all dotnet cross-compile guides I found so far. I started cross-compiling CLR and FX, but it all seems centered around ubuntu. Will the result work on angstrom, as long as the required prerequisites (like libunwind, etc) are installed? |
|
@nzain - try running using Linux (armhf) (for glibc based OS) in table on core-setup-readme. |
|
.NET Core 2.0 works flawlessly on a Raspberry Pi 3 with Raspbian Jessy Lite. The following is required to get it working: On the vanilla installation of the OS run: In your .NET Core project include: To publish on the command line/ vscode: When your project name is MyProject: copy all the files in the Publish directory to the Pi and run Enjoy :) |
|
@hqueue @jyoungyun Can we close this issue since the outstanding issue (per the list above) is in CoreCLR repo and tracked by https://github.com/dotnet/coreclr/issues/8549? |
|
@gkhanna79 Sure. I will mark https://github.com/dotnet/coreclr/issues/8549 in the list as done with comment. |
|
Closing this issue and remaining CoreCLR issues will be discussed at https://github.com/dotnet/coreclr/issues/8549 |
and others
Host: Ubuntu 14.04 x64
Target: Ubuntu 14.04 or(and) 16.04 ARM, Ubuntu 14.04 ARM Softfp and Tizen
To enable core-setup for Linux ARM32, we are going to follow steps suggested in PR dotnet/core-setup#712.
ARM hardfp
Common
System.Xml.Serialization.XmlSerializerserializes 0x00 but crashes on deserialization #1411)CoreCLR
CoreFX
core-setup
For core-setup, we will follow steps below.
For example, we want to build core-setup for Linux/ARM using following command.
ASP.NET
ARM softfp
We will use
armelforarm-softfpthrough out the dotnet.Common
System.Xml.Serialization.XmlSerializerserializes 0x00 but crashes on deserialization #1411)Debian.8
CoreFX - Part1
CoreCLR
armelforarm-softfpthrough out the dotnet. (https://github.com/dotnet/coreclr/issues/8770, Use armel instead of arm-softfp coreclr#8771)CoreFX - Part2
armelforarm-softfpthrough out the dotnet. (Use armel instead of arm-softfp corefx#14803)core-setup
For core-setup, we will follow steps below.
armelforarm-softfpthrough out the dotnet (AppDomain.AssemblyResolve can be fired/not fired depending on the type of exception thrown by other extension points #1025)Tizen 4.0.0
CoreFX - Part1
CoreCLR
CoreFX - Part2
core-setup
For core-setup, we will follow steps below.
dotnet-ci
Results
ubuntu.14.04-arm (Latest build is available at https://github.com/dotnet/core-setup#daily-builds)
dotnet-ubuntu.14.04-arm.1.2.0-beta-001291-00.tar.gz (Last updated on Jan 19)
dotnet-sdk-ubuntu.14.04-arm.1.0.0-preview5-004431.tar.gz
ubuntu.16.04-arm (Latest build is available at https://github.com/dotnet/core-setup#daily-builds)
dotnet-ubuntu.16.04-arm.1.2.0-beta-001291-00.tar.gz (Last updated on Jan 19)
dotnet-sdk-ubuntu.16.04-arm.1.0.0-preview5-004431.tar.gz
debian.8-armel
dotnet-debian.8-armel.1.2.0-beta-001271-00.tar.gz
tizen.4.0.0-armel
dotnet-tizen.4.0.0-armel.1.2.0-beta-001273-00.tar.gz
The text was updated successfully, but these errors were encountered: