Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] fix incremental builds w/ native assembler #3561

Merged
merged 1 commit into from
Aug 29, 2019

Conversation

jonathanpeppers
Copy link
Member

When testing performance, I found an issue with _GenerateJavaStubs:

  1. Change MainActivity.cs in an Android app project and hit F5.
  2. The Build runs _GenerateJavaStubs
  3. The Install runs _GenerateJavaStubs
  4. Any subsequent build (even with no changes) will run
    _GenerateJavaStubs...

The log reads:

Building target "_GenerateJavaStubs" completely.
Input file "obj\Debug\90\android\assets\App14.dll" is newer than output file "obj\Debug\90\android\typemap.jm.arm64-v8a.s".

In 74cf0ec, we modified the Outputs to be:

Outputs="$(_AndroidStampDirectory)_GenerateJavaStubs.stamp;@(_TypeMapAssemblySource)"

This would be OK, except in some cases we don't update this file:

MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName);

If _GenerateJavaStubs runs, but the typemap doesn't change, we will
be stuck in a spot where _GenerateJavaStubs will always run on
incremental builds.

_GeneratePackageManagerJava has the exact same problem as well.

IncrementalClean

The real issue 74cf0ec was trying to solve was that some of these
typemap.*.s files were getting deleted during incremental builds.

A PR build was hitting:

_CompileNativeAssemblySources:
    Assembler messages:
    can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory

Due to:

Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.

We suspected IncrementalClean was deleting these files, but we
weren't able to find the right log file where that happened...

Going through things again, I figured out what was going on:

  • @(_EnvironmentAssemblySource) and @(_NativeAssemblyTarget) were
    not added to FileWrites
  • _CreateApplicationSharedLibraries was not running during Build
    but SignAndroidPackage. This means it was running after
    IncrementalClean. Any files it produced that were in FileWrites
    are ignored...
  • We can run _CreateApplicationSharedLibraries right after
    _CompileJava. This has other benefits than correctness for
    IncrementalClean, such as when a build error occurs. The UX is a
    bit nicer in IDEs for errors from Build than Install.

After these changes I can remove the extra Outputs from
_GenerateJavaStubs and _GeneratePackageManagerJava and solely rely
on the stamp files as an output.

Moving _CreateApplicationSharedLibraries to Build should not
impact incremental build performance, as it should be target that is
skipped on almost all builds.

I also added a new test for these scenarios, verifying targets skip
and files are added to FileWrites appropriately.

Other fixes:

  • AndroidEnvironment files were not an Input to
    _GeneratePackageManagerJava. So changing environment vars seems
    like it would always require a Rebuild to work...

When testing performance, I found an issue with `_GenerateJavaStubs`:

1. Change `MainActivity.cs` in an Android app project and hit F5.
2. The `Build` runs `_GenerateJavaStubs`
3. The `Install` runs `_GenerateJavaStubs`
4. Any subsequent build (even with no changes) will run
   `_GenerateJavaStubs`...

The log reads:

    Building target "_GenerateJavaStubs" completely.
    Input file "obj\Debug\90\android\assets\App14.dll" is newer than output file "obj\Debug\90\android\typemap.jm.arm64-v8a.s".

In 74cf0ec, we modified the `Outputs` to be:

    Outputs="$(_AndroidStampDirectory)_GenerateJavaStubs.stamp;@(_TypeMapAssemblySource)"

This would be OK, except in some cases we don't update this file:

    MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName);

If `_GenerateJavaStubs` runs, but the typemap doesn't change, we will
be stuck in a spot where `_GenerateJavaStubs` will always run on
incremental builds.

`_GeneratePackageManagerJava` has the exact same problem as well.

~~ IncrementalClean ~~

The real issue 74cf0ec was trying to solve was that some of these
`typemap.*.s` files were getting deleted during incremental builds.

A PR build was hitting:

    _CompileNativeAssemblySources:
        Assembler messages:
        can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory

Due to:

    Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.

We suspected `IncrementalClean` was deleting these files, but we
weren't able to find the right log file where that happened...

Going through things again, I figured out what was going on:

* `@(_EnvironmentAssemblySource)` and `@(_NativeAssemblyTarget)` were
  not added to `FileWrites`
* `_CreateApplicationSharedLibraries` was not running during `Build`
  but `SignAndroidPackage`. This means it was running *after*
  `IncrementalClean`. Any files it produced that were in `FileWrites`
  are ignored...
* We can run `_CreateApplicationSharedLibraries` right after
  `_CompileJava`. This has other benefits than correctness for
  `IncrementalClean`, such as when a build error occurs. The UX is a
  bit nicer in IDEs for errors from `Build` than `Install`.

After these changes I can remove the extra `Outputs` from
`_GenerateJavaStubs` and `_GeneratePackageManagerJava` and solely rely
on the stamp files as an output.

Moving `_CreateApplicationSharedLibraries` to `Build` should not
impact incremental build performance, as it should be target that is
skipped on almost all builds.

I also added a new test for these scenarios, verifying targets skip
and files are added to `FileWrites` appropriately.

Other fixes:

* `AndroidEnvironment` files were not an `Input` to
  `_GeneratePackageManagerJava`. So changing environment vars seems
  like it would always require a `Rebuild` to work...
@jonpryor jonpryor merged commit 9e7b246 into dotnet:master Aug 29, 2019
@jonathanpeppers jonathanpeppers deleted the incremental-nativeassembly branch August 29, 2019 19:23
jonpryor pushed a commit that referenced this pull request Sep 4, 2019
…ler (#3561)

When testing performance, I found an issue with `_GenerateJavaStubs`:

 1. Change `MainActivity.cs` in an Android app project and hit Run.
 2. The `Build` runs `_GenerateJavaStubs`
 3. The `Install` runs `_GenerateJavaStubs`
 4. Any subsequent build (even with no changes) will run
    `_GenerateJavaStubs`...

The log reads:

	Building target "_GenerateJavaStubs" completely.
	Input file "obj\Debug\90\android\assets\App14.dll" is newer than output file "obj\Debug\90\android\typemap.jm.arm64-v8a.s".

In 74cf0ec, we modified the `Outputs` to be:

	Outputs="$(_AndroidStampDirectory)_GenerateJavaStubs.stamp;@(_TypeMapAssemblySource)"

This would be OK, except in some cases we don't update this file:

	MonoAndroidHelper.CopyIfStreamChanged (ms, asmFileName);

If `_GenerateJavaStubs` runs, but the typemap doesn't change, we will
be stuck in a spot where `_GenerateJavaStubs` will always run on
incremental builds.

`_GeneratePackageManagerJava` has the exact same problem as well.

~~ IncrementalClean ~~

The real issue 74cf0ec was trying to solve was that some of these
`typemap.*.s` files were getting deleted during incremental builds.

A PR build was hitting:

	_CompileNativeAssemblySources:
	    Assembler messages:
	    can't open typemap.mj.armeabi-v7a.s for reading: No such file or directory

Due to:

	Skipping target "_GenerateJavaStubs" because all output files are up-to-date with respect to the input files.

We suspected `IncrementalClean` was deleting these files, but we
weren't able to find the right log file where that happened...

Going through things again, I figured out what was going on:

  * `@(_EnvironmentAssemblySource)` and `@(_NativeAssemblyTarget)`
    were not added to `@(FileWrites)`
  * `_CreateApplicationSharedLibraries` was not running during `Build`
    but rather in `SignAndroidPackage`.  This means it was running
    *after* `IncrementalClean`.  Any files it produced that were in
    `@(FileWrites)` are ignored...
  * We can run `_CreateApplicationSharedLibraries` right after
    `_CompileJava`.  This has other benefits than correctness for
    `IncrementalClean`, such as when a build error occurs.  The UX is
    a bit nicer in IDEs for errors from `Build` than `Install`.

After these changes I can remove the extra `Outputs` from
`_GenerateJavaStubs` and `_GeneratePackageManagerJava` and solely
rely on the stamp files as an output.

Moving `_CreateApplicationSharedLibraries` to `Build` should not
impact incremental build performance, as it should be target that is
skipped on almost all builds.

I also added a new test for these scenarios, verifying targets skip
and files are added to `FileWrites` appropriately.

Other fixes:

  * `@(AndroidEnvironment)` files were not an `Input` to the
    `_GeneratePackageManagerJava` target.  So changing environment
    vars seemslike it would always require a `Rebuild` to work...
@github-actions github-actions bot locked and limited conversation to collaborators Jan 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants