Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] fix asset subfolders with aapt2 on Windows #2480

Merged
merged 1 commit into from
Dec 5, 2018

Conversation

jonathanpeppers
Copy link
Member

@jonathanpeppers jonathanpeppers commented Dec 3, 2018

Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=2249299&view=ms.vss-test-web.test-result-details

Since 0f91aea enabled aapt2 support by default, the following test
has been failing on Windows:

Xamarin.Android.Build.Tests.BuildAssetsTest.CheckAssetsAreIncludedInAPK / Debug
    assets/subfolder/asset2.txt should be in the apk.
    Expected: not null
    But was: null

Looking into it locally, the packaged_resources file has a weird
file path for assets in sub-directories:

assets\subfolder/asset2.txt

It appears this command line was invoked for aapt2:

Executing link
    --manifest C:\Users\myuser\Desktop\Git\xamarin-android\bin\TestDebug\temp\CheckAssetsAreIncludedInAPK\App1\obj\Release\android\manifest\AndroidManifest.xml
    --java C:\Users\myuser\AppData\Local\Temp\pjkd3dqd.4na
    --custom-package unnamedproject.unnamedproject
    -R obj\Release\flata\227168ee3afdc4839bf05d172c1a56a15a421ac7.flata
    -R obj\Release\flata\\compiled.flata
    --auto-add-overlay
    -I C:\Users\myuser\android-toolchain\sdk\platforms\android-28\android.jar
    -A C:\Users\myuser\Desktop\Git\xamarin-android\bin\TestDebug\temp\CheckAssetsAreIncludedInAPK\App1\obj\Release\assets
    -o C:\Users\myuser\Desktop\Git\xamarin-android\bin\TestDebug\temp\CheckAssetsAreIncludedInAPK\App1\obj\Release\android\bin\packaged_resources

But what makes matters worse, is the asset in the sub-directory is not
useable if the app was built on Windows!!!

Everything is pointing to a bug in aapt2 on Windows, so I filed:

https://issuetracker.google.com/issues/120436372

Changes

  • Updated CheckAssetsAreIncludedInAPK test to check aapt and aapt2.
  • Added an AssetManagerTest to Mono.Android-Tests, to verify we
    can load asset files at runtime.
  • Updated the <BuildApk/> MSBuild task to call a new
    FixupWindowsPathSeparators method on ZipArchiveEx.
  • FixupWindowsPathSeparators now looks for the \ characters in
    entries within the zip file, and swaps with the / if any are
    found.

@jonpryor
Copy link
Member

jonpryor commented Dec 3, 2018

Everything is pointing to a bug in aapt2 on Windows.

This should be validated, and an upstream bug filed if possible.

If it is an aapt2 bug on Windows, this also suggests that we should work around the issue somehow, e.g. by inserting a ".zip fixup step" (ugh) which looks for foo\bar files and replace them with foo/bar.

@jonathanpeppers jonathanpeppers added the do-not-merge PR should not be merged. label Dec 3, 2018
@jonathanpeppers
Copy link
Member Author

Bug filed for aapt2: https://issuetracker.google.com/issues/120436372

@jonathanpeppers
Copy link
Member Author

jonathanpeppers commented Dec 4, 2018

It looks like this fix is working.

The Windows failures here are tests that can randomly fail on a new build machine (I'll be looking into that later).

Let's get #2482 in and I can clean this PR up, and get it to a spot it can be merged.

jonpryor pushed a commit that referenced this pull request Dec 5, 2018
Context: #2480

The `ZipEntry.FullName` property was normalizing paths to always use
`/` as a directory separator character.  This meant that it was
not possible to [detect an `aapt2` bug][0] in which `aapt2`-generated
`.flata` files would contain `\` within the filename, e.g.

	assets\subfolder/asset1.txt

`ZipEntry.FullName` would return `assets/subfolder/asset1.txt`, and
there was no way to obtain the original/"source" value of
`assets\subfolder/asset1.txt`.

Bump to grendello/LibZipSharp/master@de1712c8 which adds a new
`ZipEntry.NativeFullName` property which contains the original
filename as present in the zip file -- `assets\subfolder/asset1.txt`
in the above example -- so that we can intelligently check for this
scenario and react accordingly in PR #2480.

Additionally, `ZipEntry.Rename()` was added, which allows renaming
a path within a zip file without needing to stream/rewrite the file
contents of that path.

[0]: https://issuetracker.google.com/issues/120436372
Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=2249299&view=ms.vss-test-web.test-result-details

Since 0f91aea enabled aapt2 support by default, the following test
has been failing on Windows:

    Xamarin.Android.Build.Tests.BuildAssetsTest.CheckAssetsAreIncludedInAPK / Debug
        assets/subfolder/asset2.txt should be in the apk.
        Expected: not null
        But was: null

Looking into it locally, the `packaged_resources` file has a weird
file path for assets in sub-directories:

    assets\subfolder/asset2.txt

It appears this command line was invoked for aapt2:

    Executing link
        --manifest C:\Users\myuser\Desktop\Git\xamarin-android\bin\TestDebug\temp\CheckAssetsAreIncludedInAPK\App1\obj\Release\android\manifest\AndroidManifest.xml
        --java C:\Users\myuser\AppData\Local\Temp\pjkd3dqd.4na
        --custom-package unnamedproject.unnamedproject
        -R obj\Release\flata\227168ee3afdc4839bf05d172c1a56a15a421ac7.flata
        -R obj\Release\flata\\compiled.flata
        --auto-add-overlay
        -I C:\Users\myuser\android-toolchain\sdk\platforms\android-28\android.jar
        -A C:\Users\myuser\Desktop\Git\xamarin-android\bin\TestDebug\temp\CheckAssetsAreIncludedInAPK\App1\obj\Release\assets
        -o C:\Users\myuser\Desktop\Git\xamarin-android\bin\TestDebug\temp\CheckAssetsAreIncludedInAPK\App1\obj\Release\android\bin\packaged_resources

But what makes matters worse, is the asset in the sub-directory is not
useable if the app was built on Windows!!!

Everything is pointing to a bug in aapt2 on Windows, so I filed:

https://issuetracker.google.com/issues/120436372

~~ Changes ~~

- Updated `CheckAssetsAreIncludedInAPK` test to check aapt and aapt2.
- Added an `AssetManagerTest` to `Mono.Android-Tests`, to verify we
  can load asset files at runtime.
- Updated the `<BuildApk/>` MSBuild task to call a new
  `FixupWindowsPathSeparators` method on `ZipArchiveEx`.
- `FixupWindowsPathSeparators` now looks for the `\` characters in
  entries within the zip file, and swaps with the `/` if any are
  found.
@jonathanpeppers jonathanpeppers changed the title [tests] workaround aapt2 failure on Windows [Xamarin.Android.Build.Tasks] fix asset subfolders with aapt2 on Windows Dec 5, 2018
@jonathanpeppers jonathanpeppers removed the do-not-merge PR should not be merged. label Dec 5, 2018
@jonathanpeppers
Copy link
Member Author

macOS test failure seems unrelated?

Test collection for System.Threading.Tasks.Tests.Status.TaskStatusTests.System.Threading.Tasks.Tests.Status.TaskStatusTests.TaskStatus10 / Release-Bundle
Error Message
Expecting Child Task final Status to be Canceled, while getting RanToCompletion\nExpected: True\nActual:   False

@jonpryor jonpryor merged commit f356220 into dotnet:master Dec 5, 2018
@github-actions github-actions bot locked and limited conversation to collaborators Feb 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants