Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profiled AOT not possible on Windows when there are spaces in project path #3407

Closed
Mikilll94 opened this issue Jul 30, 2019 · 16 comments
Closed
Assignees
Labels
Area: App+Library Build Issues when building Library projects or Application projects. bug Component does not function as intended.

Comments

@Mikilll94
Copy link

Steps to Reproduce

  1. Create a project in Xamarin.Android or Xamarin.Forms
  2. Enable profiled AOT by setting in .csproj

<AndroidEnableProfiledAot>true</AndroidEnableProfiledAot>

  1. There is an error:
    Unhandled exception: System.ArgumentException: Illegal characters in path. at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional) at System.IO.Path.GetFileName(String path) at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config) at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_01.b__0(TSource s)
    `

I think this is happening because I have some spaces in project path. If I remove spaces from the path it works.

Expected Behavior

Profile AOT should be possible even if there are spaces in the project path.

Actual Behavior

AOT is not possible because of an error

Version Information

Log File

Microsoft Visual Studio Professional 2019
Version 16.2.0
VisualStudio.16.Release/16.2.0+29123.88
Microsoft .NET Framework
Version 4.7.03056

Installed Version: Professional

Application Insights Tools for Visual Studio Package 9.1.00611.1
Application Insights Tools for Visual Studio

ASP.NET and Web Tools 2019 16.2.290.48383
ASP.NET and Web Tools 2019

ASP.NET Web Frameworks and Tools 2019 16.2.290.48383
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0 16.2.290.48383
Azure App Service Tools v3.0.0

Azure Functions and Web Jobs Tools 16.2.290.48383
Azure Functions and Web Jobs Tools

C# Tools 3.2.0-beta4-19359-03+15b43b33901c88f68ef43f8314b5a2457716780d
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools 1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Extensibility Message Bus 1.2.0 (d16-2@8b56e20)
Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.

IntelliCode Extension 1.0
IntelliCode Visual Studio Extension Detailed Info

Microsoft Azure Tools 2.9
Microsoft Azure Tools for Microsoft Visual Studio 0x10 - v2.9.20626.2

Microsoft Continuous Delivery Tools for Visual Studio 0.4
Simplifying the configuration of Azure DevOps pipelines from within the Visual Studio IDE.

Microsoft JVM Debugger 1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft Library Manager 1.0
Install client-side libraries easily to any web project

Microsoft MI-Based Debugger 1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual Studio Tools for Containers 1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Mono Debugging for Visual Studio 16.2.6 (4cfc7c3)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager 5.2.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

ProjectServicesPackage Extension 1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

ResourcePackage Extension 1.0
ResourcePackage Visual Studio Extension Detailed Info

ResourcePackage Extension 1.0
ResourcePackage Visual Studio Extension Detailed Info

SQL Server Data Tools 16.0.61906.28070
Microsoft SQL Server Data Tools

Syntax Visualizer 1.0
An extension for visualizing Roslyn SyntaxTrees.

TypeScript Tools 16.0.10627.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools 3.2.0-beta4-19359-03+15b43b33901c88f68ef43f8314b5a2457716780d
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 10.4 for F# 4.6 16.2.0-beta.19321.1+a24d94ecf97d0d69d4fbe6b8b10cd1f97737fff4
Microsoft Visual F# Tools 10.4 for F# 4.6

Visual Studio Code Debug Adapter Host Package 1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for Containers 1.0
Visual Studio Tools for Containers

VisualStudio.Mac 1.0
Mac Extension for Visual Studio

Xamarin 16.2.0.90 (d16-2@ba267630e)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 16.2.0.325 (remotes/origin/d16-2@f10cfbf83)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin Templates 16.3.117 (59a59e8)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.

Xamarin.Android SDK 9.4.0.51 (d16-2/9fa7775)
Xamarin.Android Reference Assemblies and MSBuild support.
Mono: mono/mono@e6f5369c2d2
Java.Interop: xamarin/java.interop/d16-2@d64ada5
LibZipSharp: grendello/LibZipSharp/d16-2@caa0c74
LibZip: nih-at/libzip@b95cf3f
ProGuard: xamarin/proguard@905836d
SQLite: xamarin/sqlite@8212a2d
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-2@6f6c969

Xamarin.iOS and Xamarin.Mac SDK 12.14.0.110 (a8bcecc)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

@jpobst jpobst added Area: App+Library Build Issues when building Library projects or Application projects. bug Component does not function as intended. labels Jul 31, 2019
@jonathanpeppers
Copy link
Member

@Mikilll94 can you post a diagnostic MSBuild log of the failure?

I tried testing this scenario, but using a path with spaces was working for me: master...jonathanpeppers:profiledaot-spaces

@Mikilll94
Copy link
Author

Mikilll94 commented Aug 1, 2019

Here it is:

(...)
[AOT] MONO_PATH="D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets" MONO_ENV_OPTIONS="" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Xamarin\Android\cross-arm" --aot=profile-only,profile=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android\startup-xf.aotprofile,msym-dir=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a,outfile=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\libaot-Xamarin.Android.Support.Design.dll.so,asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\PROGRA~2\Android\ANDROI~1\NDK-BU~1\TOOLCH~1\ARM-LI~1.9\prebuilt\WINDOW~1\bin\arm-linux-androideabi-,ld-flags=,llvm-path=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android,temp-path=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\Xamarin.Android.Support.Design.dll "D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets\Xamarin.Android.Support.Design.dll"
[AOT] MONO_PATH="D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets" MONO_ENV_OPTIONS="" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Xamarin\Android\cross-arm" --aot=profile-only,profile=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android\startup-xf.aotprofile,msym-dir=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a,outfile=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\libaot-Xamarin.Android.Support.Fragment.dll.so,asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\PROGRA~2\Android\ANDROI~1\NDK-BU~1\TOOLCH~1\ARM-LI~1.9\prebuilt\WINDOW~1\bin\arm-linux-androideabi-,ld-flags=,llvm-path=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android,temp-path=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\Xamarin.Android.Support.Fragment.dll "D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets\Xamarin.Android.Support.Fragment.dll"
[AOT] MONO_PATH="D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets" MONO_ENV_OPTIONS="" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Xamarin\Android\cross-arm" --aot=profile-only,profile=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android\startup-xf.aotprofile,msym-dir=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a,outfile=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\libaot-Xamarin.Android.Support.Media.Compat.dll.so,asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\PROGRA~2\Android\ANDROI~1\NDK-BU~1\TOOLCH~1\ARM-LI~1.9\prebuilt\WINDOW~1\bin\arm-linux-androideabi-,ld-flags=,llvm-path=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android,temp-path=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\Xamarin.Android.Support.Media.Compat.dll "D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets\Xamarin.Android.Support.Media.Compat.dll"
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
[aot-compiler stderr] Unknown command line option: '-'
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
[AOT] MONO_PATH="D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets" MONO_ENV_OPTIONS="" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Xamarin\Android\cross-arm" --aot=profile-only,profile=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android\startup-xf.aotprofile,msym-dir=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a,outfile=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\libaot-Xamarin.Android.Support.Transition.dll.so,asmwriter,mtriple=armv7-linux-gnueabi,tool-prefix=C:\PROGRA~2\Android\ANDROI~1\NDK-BU~1\TOOLCH~1\ARM-LI~1.9\prebuilt\WINDOW~1\bin\arm-linux-androideabi-,ld-flags=,llvm-path=C:\PROGRA~2\MICROS~4\2019\PROFES~1\MSBuild\Xamarin\Android,temp-path=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\Xamarin.Android.Support.Transition.dll "D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\android\assets\Xamarin.Android.Support.Transition.dll"
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
XA0000: Unhandled exception: System.ArgumentException: Illegal characters in path.
   at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional)
   at System.IO.Path.GetFileName(String path)
   at Xamarin.Android.Tasks.Aot.<>c__DisplayClass77_0.<RunParallelAotCompiler>b__0(Config config)
   at Xamarin.Android.Tasks.AsyncTaskExtensions.<>c__DisplayClass0_0`1.<ParallelForEach>b__0(TSource s)
Done executing task "Aot" -- FAILED.
Done building target "_BuildApkEmbed" in project "MyApp.Android.csproj" -- FAILED.
Done building project "MyApp.Android.csproj" -- FAILED.
Build FAILED.
3>Deployment failed to nexus_5x.
========== Build: 2 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
========== Deploy: 0 succeeded, 1 failed, 0 skipped ==========

@dellis1972
Copy link
Contributor

This is the old Escaping issue we always have when calling mono on windows :(
In this case temp-path

temp-path=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\Xamarin.Android.Support.Fragment.dll

However this is already using the GetShortPath call.. https://github.com/xamarin/xamarin-android/blob/master/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs#L433

Which should be returning a short path for those bits with spaces etc.

@dellis1972 dellis1972 added this to the d16-4 milestone Aug 1, 2019
@jonathanpeppers
Copy link
Member

Hmm, when I looked at a log of mine that is working, it did this:

temp-path=C:\src\XAMARI~1\bin\TESTDE~1\temp\BUILDA~1\obj\Release\aot\ARMEAB~1\UnnamedProject.dll

So why isn't that happening here?

@Mikilll94 that build log doesn't look like it has diagnostic verbosity, can you turn that on and send a new log?
https://docs.microsoft.com/en-us/xamarin/android/troubleshooting/troubleshooting#diagnostic-msbuild-output

Or if you can get an MSBuild binlog that is even better:
https://github.com/microsoft/msbuild/blob/master/documentation/wiki/Binary-Log.md

@Mikilll94
Copy link
Author

Mikilll94 commented Aug 2, 2019

@jonathanpeppers

Hey,
this is a log with a diagnostic verbosity.

Look at my temp-path:

temp-path=D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a\Xamarin.Android.Support.Transition.dll

As you can see my project is localized in "MyApp - new". And this directory name has 2 SPACES.

In temp-path which you have provided, I don't see any spaces. So maybe that's the reason that it is working for your project.

@jonathanpeppers
Copy link
Member

@Mikilll94 a diagnostic log would be enormous, it looks like you may have posted a snippet of one? You should be able to zip up the full log and attach in a Github comment.

Two spaces also works for me, which is why we need a full log to figure out the issue here.

@Mikilll94
Copy link
Author

Mikilll94 commented Aug 6, 2019

@jonathanpeppers

Here is a full log:

build_log.txt

@jonathanpeppers
Copy link
Member

@Mikilll94 reviewing what we do here: https://github.com/xamarin/xamarin-android/blob/ecc016a2157e2a048dfd08f57e5ce4952aae3127/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs#L176-L182

What path does your machine report if you run this sample? ShortPath.zip

Mine outputs:

ShortPath: C:\Projects\MyApp\MYAPP-~1\MyApp\MYAPP~1.AND\obj\RELEAS~1\81\aot\ARMEAB~1

I tried C:\ but try D:\ on your machine too if you can, thanks!

@Mikilll94
Copy link
Author

Mikilll94 commented Aug 8, 2019

@jonathanpeppers

For my C drive it prints:
ShortPath: C:\Projects\MyApp\MYAPP-~1\MyApp\MYAPP~1.AND\obj\RELEAS~1\81\aot\ARMEAB~1

For my D drive it prints:
ShortPath: D:\Projects\MyApp\MyApp - new\MyApp\MyApp.Android\obj\ReleaseTest\81\aot\armeabi-v7a

Try also on your computer. You should have the same results.

@kvpt
Copy link

kvpt commented Aug 8, 2019

GetShortPathName is based on a 8dot3 parameter.
By default it's only activated on system partition.

It's possible that for certains users all partitions have this parameter activated so it always work.

You can check it by running this command.

fsutil 8dot3name query c:

fsutil 8dot3name query d:

@Mikilll94
Copy link
Author

Mikilll94 commented Aug 8, 2019

@kvpt

You are right!

I have checked these two commands.

On my computer 8.3 short file names are only enabled on C: drive. On D: drive short filenames are disabled.

@jonathanpeppers
Copy link
Member

We really need to just kill these 8dot3 paths and not use them at all, the issue was that we didn't have a way to run the Mono AOT compilers with a "response file" as input.

We started this effort here: #2062

If I remember correctly, we needed some further changes on the Mono side. @dellis1972 do you remember?

@dellis1972
Copy link
Contributor

@jonathanpeppers there still needs to be some changes on the mono side. If I remember rightly it was not parsing the response file correctly and did not deal will spaces correctly either.

@ElVinche
Copy link

ElVinche commented Aug 14, 2019

Hello, i confirmed i have the same trouble with my computer. The beginning of the path of the directories of all my projects is "E:\visual studio 2015\Projects..." and i have the error "[aot-compiler stderr] Cannot open assembly 'studio': No such file or directory." On E: volume, short file names isn't enabled only on C:.
Same behaviors on VS 2019 16.2.2 or VS 2019 16.3 Preview 2.0.

Thanks for your help.

jonpryor pushed a commit that referenced this issue Sep 6, 2019
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/609244
Fixes: https://developercommunity.visualstudio.com/content/problem/554407/vs2019-xamarinandroid-aot-command.html
Fixes: #3407

Our AOT system has issues when a user uses non-ASCII characters and
spaces on Windows.  We used [`GetShortPathName()`][0] to get the old
DOS 8.3 short names of paths to get around paths having spaces and
unicode characters.

However, short path name generation can be [disabled][1], in which
case `GetShortPathName()` will return the long path name.
Consequently, builds can fail when spaces appear in unexpected places:

	[aot-compiler stderr] Cannot open assembly 'Files': No such file or directory.	

Short path name generation can be disabled on a drive-by-drive basis,
and our Azure DevOps build machines have short path name generation
disabled on the `E:` drive used for builds.

We really need to support paths with spaces and unicode characters.

Rework the way we provide the `--aot` argument to the cross compilers
so that it actually works in those scenarios.

The first thing was how the arguments were parsed.  `mono` uses the
built-in system command line parser [`CommandLineToArgv()`][2] to
parse arguments.  Given the following `--aot` argument

	--aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so",asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp"

This ends up as the following arguments, one per-line

	[0]: --aot=outfile="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle
	[1]: AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so"
	[2]: ,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path="c:\Sandbox\repo\BuildAotApplicationAndBundle
	[3]: AndÜmläüts_x86_True_True\obj\Release\temp"

As you can see the parameters have been split wherever there is a
space.  The solution to this is to double quote the ENTIRE argument
and remove any quotes within the parameter list like:

	"--aot=outfile=c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=c:\Sandbox\repo\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\temp"

This allows Windows (and mac) to parse the parameters correctly as
one value.

There is another issue however.  With the new argument line above, if
the `temp-path=` path has a space in it then `mono` still has issues
with that path.  The good news is that we can use some domain
knowledge to reduce not only the paths which need spaces but also the
overall length of the argument.  Because we know the cross compiler
will be executed within `WorkingDirectory` we can shorten any path
which is within that directory structure.  `WorkingDirectory` is set
to the directory of the projects `.csproj` file.  So the following:

	E:\Some Project\My Project\obj\Release\aot\System.dll\

will become

	obj\Release\aot\System.dll\

This will fix the issue with the `temp-path=` argument.  We end up
with something like this

	"--aot=outfile=obj\Release\libaot-Mono.Android.dll.so,asmwriter,mtriple=i686-linux-android,tool-prefix=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\bin\i686-linux-android-,ld-flags=C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\toolchains\x86-4.9\prebuilt\windows\lib\gcc\i686-linux-android\4.9.x\libgcc.a;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libc.so;C:\ProgramData\Microsoft\AndroidNDK\android-ndk-r13b\platforms\android-16\arch-x86\usr\lib\libm.so,temp-path=obj\Release\temp"

However we might also still start hitting the
[command line length limit][3] on Windows.  This is currently 8191
characters on Windows XP and later.  So depending on where a user
creates the project we might end up with a "too long" command line.

To work around this issue we can make use of the new `mono --response=`
argument.  Instead of passing all the arguments to the cross compiler
we can instead write them to a file and pass the path to that file as
the `--response=` argument.  This will reduce our command line length
to be within an acceptable range unless the user creates a project in
a very very deep directory structure.  The final call will be like:

	...\cross-arm --response="c:\Sandbox\repo\bin\BuildAotApplicationAndBundle AndÜmläüts_x86_True_True\obj\Release\Mono.Android.dll\response.txt"

which works perfectly.

This commit also updates the `BuildTest.BuildAotApplication()` and
`BuildTest.BuildAotApplicationAndBundle()` unit tests to use paths
with both spaces and non-ASCII characters to ensure we support both
of those scenarios.

[0]: https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getshortpathnamew
[1]: https://support.microsoft.com/en-us/help/121007/how-to-disable-8-3-file-name-creation-on-ntfs-partitions
[2]: https://docs.microsoft.com/en-us/windows/win32/api/shellapi/nf-shellapi-commandlinetoargvw
[3]: https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation
@brendanzagaeski
Copy link
Contributor

Release status update

A new Preview version has now been published that includes the fix for this item. The fix is not yet included in a Release version. I will update this item again when a Release version is available that includes the fix.

Fix included in Xamarin.Android 10.0.99.100.

Fix included on Windows in Visual Studio 2019 version 16.4 Preview 1. To try the Preview version that includes the fix, check for the latest updates in Visual Studio Preview.

@brendanzagaeski
Copy link
Contributor

Release status update

A new Release version has now been published that includes the fix for this item.

Fix included in Xamarin.Android 10.1.0.30.

Fix included on Windows in Visual Studio 2019 version 16.4. To get the new version that includes the fix, check for the latest updates or install the latest version from https://visualstudio.microsoft.com/downloads/.

@ghost ghost locked as resolved and limited conversation to collaborators Jun 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Area: App+Library Build Issues when building Library projects or Application projects. bug Component does not function as intended.
Projects
None yet
Development

No branches or pull requests

7 participants