Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARM 32-bit progress #5460

Closed
manu-st opened this issue Mar 30, 2016 · 123 comments
Closed

ARM 32-bit progress #5460

manu-st opened this issue Mar 30, 2016 · 123 comments
Labels
Milestone

Comments

@manu-st
Copy link

@manu-st manu-st commented Mar 30, 2016

I'm opening an issue to track the progress of ARM 32-bit with respects to the regression tests.

Currently I'm getting the following results on Ubuntu 14.04 running on a NVidia Jetson TK1. This is unfortunately not the results you get from a clean checkout, the only difference is that it contains a fix for issue #5422 (PR dotnet/coreclr#3879), but overall it should provide a good base to monitor our progress:

=======================
     Test Results
=======================
# Tests Discovered : 6018
# Passed           : 4313
# Failed           : 1361
# Skipped          : 344
=======================

I've attached the XML results:
coreclrtests.zip

@leemgs
Copy link

@leemgs leemgs commented Mar 30, 2016

This is unfortunately not the results you get from a clean checkout,

Even though you could get the 1,361 Failed result, this result is meaningful to me. Actually, we have been meeting a lot of "NYI" functions from CoreCLR.:( It means that we have to do a lot of things for stablization of CoreCLR.

Currently I'm getting the following results on Ubuntu 14.04 running on a NVidia Jetson TK1

BTW, Do you also focus on the Ubuntu/ARM32bit without the Ubuntu/ARM64bit?
http://elinux.org/Jetson_TK1

  • CPU: NVIDIA "4-Plus-1" 2.32GHz ARM quad-core Cortex-A15 CPU with Cortex-A15 battery-saving shadow-core
  • GPU: NVIDIA Kepler "GK20a" GPU with 192 SM3.2 CUDA cores (upto 326 GFLOPS)
  • Memory: ~ 8GiB DDR3

\CC: @myungjoo , @lemmaa
They also interested in the regression test of CoreCLR.

@manu-st
Copy link
Author

@manu-st manu-st commented Mar 30, 2016

@leemgs I'm not sure to understand what you mean by Ubuntu/ARM32bit without the Ubuntu/ARM32bit part?

@leemgs
Copy link

@leemgs leemgs commented Mar 30, 2016

@manu-silicon Sorry, Typo, I have modified.

@manu-st
Copy link
Author

@manu-st manu-st commented Mar 30, 2016

@leemgs My board is the 32-bit variant, so it is just Ubuntu/ARM32bit.

https://en.wikipedia.org/wiki/Tegra#Tegra_K1

@manu-st
Copy link
Author

@manu-st manu-st commented Mar 31, 2016

With the fix for PR dotnet/coreclr#3981 the result is:

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 4793
# Failed           : 882
# Skipped          : 344
=======================

coreclrtests.zip

@prajwal-aithal
Copy link

@prajwal-aithal prajwal-aithal commented Mar 31, 2016

@manu-silicon I am trying to do the same (run coreclr tests on ARM environment) by following this guide https://github.com/dotnet/coreclr/blob/master/Documentation/building/unix-test-instructions.md. According to this, building tests for ARM results in an error (as building the native code using cmake in Windows does not support arm architecture yet). I can make some changes to the buildtest script to build only the C# based tests for ARM architecture.

I wanted to ask how you are building the test binaries for ARM environment (using buildtest.cmd). Have you made some local changes (to build only the C# code and ignore the CMake build) or is there some other way to do it?

@manu-st
Copy link
Author

@manu-st manu-st commented Mar 31, 2016

@prajwal-aithal I've followed the instructions and build the tests on Windows and copy the tests folder on Linux ARM. I built on ARM the native code for CoreFX (just the native code as the rest won't build). I built CoreFX on Linux x64 and copied the assemblies over. I built mscorlib on Linux x64 and copied it over.

Once I had all the paths setup properly, I launched the command to run the tests.

@prajwal-aithal
Copy link

@prajwal-aithal prajwal-aithal commented Mar 31, 2016

@manu-silicon Oh, ok. I will try this then. Thank you :)

@manu-st
Copy link
Author

@manu-st manu-st commented Mar 31, 2016

Today's result with FEATURE_STUBS_AS_IL disabled:

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 4836
# Failed           : 839
# Skipped          : 344
=======================

coreclrtests.zip

@jyoungyun
Copy link

@jyoungyun jyoungyun commented Apr 12, 2016

Target: Raspberry Pi2
CoreClr commit id : ff26d68 (around april 9)

=======================
     Test Results
=======================
# Tests Discovered : 7422
# Passed           : 6074
# Failed           : 1017
# Skipped          : 331
=======================

I'm curious why my test discovered count is different with others.
CoreClr_UnitTest_Results_160412_01.zip

@myungjoo
Copy link

@myungjoo myungjoo commented Apr 12, 2016

@jyoungyun Next time, please attach the result XML file as well.

@manu-st
Copy link
Author

@manu-st manu-st commented Apr 12, 2016

@jyoungyun The difference in the number of tests is related on how I build first the test on Windows and then copied them over to the Linux ARM device. As @myungjoo is saying, having the XML file would enable us to compare and see what the differences are.

@jyoungyun
Copy link

@jyoungyun jyoungyun commented Apr 19, 2016

@manu-silicon I attached xml file. Thank you.

@jyoungyun
Copy link

@jyoungyun jyoungyun commented Apr 27, 2016

Target: Raspberry Pi2
Coreclr commit id : 31fada1 + dotnet/coreclr#4460 + dotnet/coreclr#4503

==========================
      Test Results
==========================
# Tests Discovered : 7421
# Passed           : 6547
# Failed           : 543
# Skipped          : 331
==========================

CoreClr_UnitTest_Results_160427.zip

@masonwheeler
Copy link

@masonwheeler masonwheeler commented Apr 27, 2016

Wow, so those two fixes cleared up about half of the failing tests? Nice!

@jyoungyun
Copy link

@jyoungyun jyoungyun commented Apr 28, 2016

@masonwheeler Yes! The dotnet/coreclr#4460 patch reduced half of failing tests. :)

@jyoungyun
Copy link

@jyoungyun jyoungyun commented Apr 28, 2016

Target: Raspberry Pi2
Coreclr commit id: 18268be + dotnet/coreclr#4503 + dotnet/coreclr#4581

==========================                                                      
      Test Results                                                              
==========================                                                      
# Tests Discovered : 6018                                                       
# Passed           : 5295                                                       
# Failed           : 395                                                        
# Skipped          : 328                                                        
==========================  

And I found the reason why my tests discovered count is different with others.
My test directory structure was wrong. Now it's the same with others. :)

CoreClr_UnitTest_Results_160428.zip

@leemgs
Copy link

@leemgs leemgs commented Apr 28, 2016

@jyoungyun , It's amazing. : ) BTW, Is dotnet/coreclr@18268be related to the number of the reduced failure? It seems that it is not related to the failure numbers of the unit test for Linux/ARM.

@manu-st
Copy link
Author

@manu-st manu-st commented Apr 28, 2016

@jyoungyun If possible, it would be nice to know how many tests each bug fix addresses.

@manu-st
Copy link
Author

@manu-st manu-st commented Apr 28, 2016

@jyoungyun Out of curiosity, how long does it take you to run all the tests. For me on the Jetson TK1, it takes roughly half a day.

@myungjoo
Copy link

@myungjoo myungjoo commented Apr 28, 2016

@manu-silicon We will be pushing out patches that accelerate the execution of corerun at ARM soon. @leemgs is preparing PR for that (release build bugfix)

@leemgs
Copy link

@leemgs leemgs commented Apr 28, 2016

For reference, @manu-silicon has shared the hardware specification that he used for getting the result of unit test. Here is the detail contents of his development board.

@jyoungyun
Copy link

@jyoungyun jyoungyun commented Apr 28, 2016

@leemgs Oh, The number(18268be) means the base commit. Someone want to reproduce the result, the base commit info. will be helpful. Reducing failure test cases are more to do with dotnet/coreclr#4460 patch, I think.
@manu-silicon It takes a lot of time to run unix test on Raspberry pi2. In my case, about 12 hours is the usaual time required. But when I use a release mode binary, it takes only 4 hours! I hope that @leemgs patch will be ready soon.

@hqueue
Copy link
Member

@hqueue hqueue commented May 3, 2016

In the lastet commit(92d7091), I got much better result. I was investigating several failures and many of them are fixed in the latest master branch.
Can anybody run the whole test again?

@manu-st
Copy link
Author

@manu-st manu-st commented May 4, 2016

@hqueue I can do it on Friday.

@manu-st
Copy link
Author

@manu-st manu-st commented May 9, 2016

Tried to run the test suite on Friday, but it got stuck on 2 tests that I had to manually kill:

FAILED   - JIT/Regression/CLR-x86-JIT/V2.0-Beta2/b425314/b425314/b425314.sh
               BEGIN EXECUTION
               /home/ubuntu/local/Microsoft/tests/Tests/coreoverlay/corerun b425314.exe
               ./b425314.sh: line 62: 32428 Killed                  $_DebuggerFullPath "$CORE_ROOT/corerun" b425314.exe $CLRTestExecutionArguments $Host_Args
               Expected: 100
               Actual: 137
               END EXECUTION - FAILED

FAILED   - JIT/Regression/CLR-x86-JIT/V2.0-Beta2/b426654/b426654/b426654.sh
               BEGIN EXECUTION
               /home/ubuntu/local/Microsoft/tests/Tests/coreoverlay/corerun b426654.exe
               ./b426654.sh: line 62: 10285 Killed                  $_DebuggerFullPath "$CORE_ROOT/corerun" b426654.exe $CLRTestExecutionArguments $Host_Args
               Expected: 100
               Actual: 137
               END EXECUTION - FAILED
@manu-st
Copy link
Author

@manu-st manu-st commented May 9, 2016

The run finally completed. Results on Jetson TK1 commit 3ddd9c4

=======================
     Test Results
=======================
# Tests Discovered : 6019
# Passed           : 5478
# Failed           : 234
# Skipped          : 307
=======================

coreclrtests_2016_05_06.zip

@jyoungyun
Copy link

@jyoungyun jyoungyun commented May 10, 2016

The latest unit test results!

Target: Raspberry Pi2
Coreclr commit id: 6b92cff

==========================
      Test Results
==========================
# Tests Discovered : 6530
# Passed           : 5981
# Failed           : 215
# Skipped          : 334
==========================

CoreClr_UnitTest_Results_160510.zip

@manu-silicon When I run unix tests, some tc takes such a long time. And the blocked tc is changed. Have you seen the same case ?

@myungjoo
Copy link

@myungjoo myungjoo commented May 10, 2016

Note that we have about 120 fails due to regression being hopefully fixed easily with dotnet/coreclr#4888. We may see under 100 fails soon.

@manu-st
Copy link
Author

@manu-st manu-st commented May 10, 2016

@jyoungyun I'm not sure what you mean by "tc is changed". But as I said it just before posting my results, I had 2 tests that was hanging preventing the rest of the test cases to run. I had to kill them manually to get to the completion.

@jaestevan
Copy link

@jaestevan jaestevan commented Mar 30, 2017

Then is Ubuntu in the Pi the only way to run the project? I was guessing it can be ran in Windows 10 IoT too.

@Petermarcu
Copy link
Member

@Petermarcu Petermarcu commented Mar 30, 2017

I'm curious what you hit on Win10 IoT? You shouldn't need to install any core runtime. The instruction put the runtime with your application.

@darxis
Copy link
Collaborator

@darxis darxis commented Mar 30, 2017

In case it would help anyone, I'm posting my instruction notes on how to install the dotnet tool on RPi3 to run .net core apps.

  1. Clone coreclr repo
  2. Compile coreclr
    1. Install everything from https://github.com/dotnet/coreclr/blob/master/Documentation/building/windows-instructions.md
    2. build.cmd arm release skiptests skipbuildpackages
  3. Clone corefx repo
  4. Compile corefx
    1. build-native.cmd -release -buildArch=arm
    2. build-managed.cmd -release -buildArch=arm -skiptests
  5. Clone core-setup repo
  6. Compile core-setup
    1. build.cmd -Configuration Release -TargetArch arm
  7. Copy output files to Windows 10 IoT
  8. Get-ChildItem "U:\netcore\corefx" -Directory | ForEach-Object { Copy-Item $("U:\netcore\corefx{0}{0}.dll" -f $_.name) "U:\netcore\corefx" }
  9. Get-ChildItem "U:\netcore\corefx" -Directory | ForEach-Object {Write-Host $_.name } Remove-Item -recurse

With this I got working an .NET Core console app and an ASP.NET Core web app that is running 24/7 for about 2 months.

EDIT: My notes were taken when there were no official instructions on how to install on RPi3. Consider first trying these ones https://github.com/dotnet/core/blob/master/samples/RaspberryPiInstructions.md

@Gonkers
Copy link

@Gonkers Gonkers commented Mar 30, 2017

Sorry @jaestevan I should have been more clear. I meant to point out the screen shots of how/were you can update the framework version. To see if that would help you get it running. That was my biggest hang up for a while, figuring out building for the correct version of the framework.

@jaestevan
Copy link

@jaestevan jaestevan commented Mar 30, 2017

@Petermarcu by following this instructions I got a CLR error: Failed to initialize CoreCLR, HRESULT: 0X80131534

@Gonkers oh, I see what you mean. Thanks.

@darxis I will try to build it myself when I have time, thanks.

@ALL: Found this link: http://developers.de/blogs/damir_dobric/archive/2015/09/22/how-to-run-net-core-application-on-pi2-win10-iot-core.aspx Still have not tried it myself but I will. Anyway I guess it should be a cleaner way when ARM support is official.

@Petermarcu
Copy link
Member

@Petermarcu Petermarcu commented Mar 30, 2017

@jaestevan , I'll give it another try, hadn't done it on Win10 IoT in a few weeks. You shouldn't have to build the full stack but its always an option.

@jaestevan
Copy link

@jaestevan jaestevan commented Mar 30, 2017

@Petermarcu Thanks Peter, let me know how it works. I can send you the details I followed if that helps.

@Petermarcu
Copy link
Member

@Petermarcu Petermarcu commented Mar 31, 2017

I tried it again yesterday and everything worked really well.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
    <RuntimeIdentifiers>win10-arm</RuntimeIdentifiers>
  </PropertyGroup>
</Project>

dotnet restore
dotnet publish -r win10-arm

Then I copy bin\Debug\netcoreapp2.0\win10-arm\publish\* to the Pi and execute .\Helloworld.exe

I have one guess as to what is happening. Can you run dotnet --version to make sure you are on a 2.0.0 SDK and make sure you have the latest one? What I do for the pre-release SDK's is I extract the zip to some location, open up a command windows, make sure the path to where I exctracted the zip is the first thing on my path, and then run dotnet commands. Right now, 1.0.0 is winning over 2.0.0 because of an issue in the SDK that is going to be fixed in the next couple weeks.

@nrandell
Copy link

@nrandell nrandell commented Apr 10, 2017

I've just been experimenting with this and have it working with SDK version 2.0.0-preview1-005743 with runtime version 2.0.0-preview1-001915-00.

I've been able to edit and build using that latest Visual Studio 2017! All looking really good.

I've got a repository at https://github.com/sceneskope/PiDotnet which has some projects where this works. I'm actually using docker as containers for my images as the projects I'm working on have to run on multiple PIs.

@TAGC
Copy link

@TAGC TAGC commented Apr 10, 2017

I've also created a .NET Core app that runs on an RPI 3 using SDK 2.0.0-preview1-005685 and runtime framework 2.0.0-beta-001834-00.

Runs fine natively (although I get exec format errors if trying to run it within a Docker container based on microsoft/dotnet-nightly:2.0.0-beta-runtime).

@nrandell
Copy link

@nrandell nrandell commented Apr 10, 2017

The Microsoft images won't be for the ARM architecture, which is why I've created my own set of docker images! It would be great to have proper Microsoft images though.

@TAGC
Copy link

@TAGC TAGC commented Apr 10, 2017

@nrandell Oh nice, I missed that. I'll have to try it out at some point.

@MihaMarkic
Copy link

@MihaMarkic MihaMarkic commented Apr 11, 2017

@nrandell Is that a fact written down somewhere? I mean no ARM docker images from MS.

@nrandell
Copy link

@nrandell nrandell commented Apr 11, 2017

@MihaMarkic have a look at dotnet/dotnet-docker#223. I guess this is as official as it gets.

@MihaMarkic
Copy link

@MihaMarkic MihaMarkic commented Apr 12, 2017

@nrandell So, they'll provide one for .net core 2.0 eventually but skip 1.x.

@jaestevan
Copy link

@jaestevan jaestevan commented Apr 14, 2017

Upgraded and rebuilt everything with 2.0.0-preview1-001978-00 and now it works in the Pi 💃

@AndreiRinea
Copy link

@AndreiRinea AndreiRinea commented Apr 17, 2017

Excuse me if this has been discussed before, but will this run on Raspberry Pi Zero too?

@benpye
Copy link

@benpye benpye commented Apr 17, 2017

@wanton7
Copy link

@wanton7 wanton7 commented Apr 17, 2017

I think pi 1 and zero have ARMv6k that has some enhancements compared to ARMv6.

@masonwheeler
Copy link

@masonwheeler masonwheeler commented Apr 17, 2017

@wanton7 Hoo boy, even more versionitis for us to keep straight?

@wanton7
Copy link

@wanton7 wanton7 commented Apr 17, 2017

@masonwheeler not sure what the differences are but example next golang (1.9 i think) is going to still support ARMv6k but they are dropping support for ARMv6. So must be something significant.

@BruceForstall
Copy link
Member

@BruceForstall BruceForstall commented Apr 25, 2018

@Petermarcu I propose to close this issue; we're far past the "ARM 32-bit progress" title -- we have many thousands of tests running on ARM Linux in our CI. Agreed?

@myungjoo
Copy link

@myungjoo myungjoo commented Apr 25, 2018

I agree. ;)

@Petermarcu Petermarcu closed this Apr 25, 2018
@msftgits msftgits transferred this issue from dotnet/coreclr Jan 30, 2020
@msftgits msftgits added this to the Future milestone Jan 30, 2020
@msftbot msftbot bot locked as resolved and limited conversation to collaborators Jan 1, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.