Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Marshal methods part 4 of 5 (runtime) #7285

Merged
merged 60 commits into from
Sep 7, 2022
Merged

Commits on Jul 26, 2022

  1. Implement marshal methods LLVM IR executable code generator.

    The point of this commit is only to generate code and make sure it's
    valid as far as compiling and linking are concerned. The code has not
    been tested at run time as not all the infrastructure on the
    Xamarin.Android side is implemented yet. This is on purpose, to keep PRs
    smaller.
    
    The majority of this PR introduces various classes, enums and structures
    related to code generation. Support for various LLVM IR instructions is
    limited only to those we actually use and only to elements of those
    constructions that we use. As such, it's not a general purpose code
    generator which allows us to make some assumptions and take some
    shortcuts (without compromising correctness and validity of the
    generated code)
    
    Portions of the PR (the native type handling system) are to be treated
    as proof-of-concept as they are not as optimized (design wise) as they
    should be. The reason for this limitation is that it requires modifying
    the previous LLVM IR data generation code and it would contribute to
    this PR's already substantial size. The next PR in the series will take
    care of that rewrite as well as it will focus on implementing the
    runtime side of marshal methods, making it possible to actually run
    applications which use marshal methods.
    
    What this PR implements is the following:
    
      * LLVM IR
        * function and instruction attributes
        * function parameter (declaration/definition) and argument (runtime)
          handling
        * function variable (including parameters) handling, including
          unnamed local variables
        * support for native function signatures and pointers to functions
        * The ret, store, load, icmp, br, call and phi instructions
      * Marshal method generator
        * managed to JNI signature and symbol name translations
    grendello committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    35f4e24 View commit details
    Browse the repository at this point in the history
  2. [marshal methods] Runtime fixes and missing features

    Implement missing runtime support so that applications are able to
    actually run.  Also fix issues with marshal method overloads as well as
    add code to handle the case where two unrelated methods (from different
    types) end up using the same JNI symbol name:
    
      * `Java_crc64e1fb321c08285b90_CellAdapter_n_1onActionItemClicked
        * `System.Boolean Android.Views.ActionMode/ICallback::OnActionItemClicked(Android.Views.ActionMode,Android.Views.IMenuItem)`
        * `System.Boolean AndroidX.AppCompat.View.ActionMode/ICallback::OnActionItemClicked(AndroidX.AppCompat.View.ActionMode,Android.Views.IMenuItem)`
    grendello committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    e34f88e View commit details
    Browse the repository at this point in the history
  3. Update

    grendello committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    eb15bd2 View commit details
    Browse the repository at this point in the history
  4. TODOs and CWLs

    grendello committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    a80e4a1 View commit details
    Browse the repository at this point in the history
  5. Update JI

    grendello committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    49751f7 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2022

  1. Merge branch 'main' into mm-codegen

    * main:
      LEGO: Merge pull request 7221
      LEGO: Merge pull request 7219
      [Xamarin.Android.Build.Tasks] use `ToJniName(type, cache)` (dotnet#7211)
      [docs] Synchronize with MicrosoftDocs/xamarin-docs (dotnet#7208)
      [Mono.Android] Remove System.Linq usage (dotnet#7210)
      Bump to Android NDK r25 (dotnet#6764)
      Bump to mono/opentk@daa9b2d5 (dotnet#7192)
      [Mono.Android] Optional NTLMv2 support in AndroidMessageHandler (dotnet#6999)
      Bump to dotnet/installer@53587f9 7.0.100-rc.1.22374.1 (dotnet#7198)
    grendello committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    803b3e3 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      LEGO: Merge pull request 7221
      LEGO: Merge pull request 7219
      [Xamarin.Android.Build.Tasks] use `ToJniName(type, cache)` (dotnet#7211)
      [docs] Synchronize with MicrosoftDocs/xamarin-docs (dotnet#7208)
      [Mono.Android] Remove System.Linq usage (dotnet#7210)
      Bump to Android NDK r25 (dotnet#6764)
      Bump to mono/opentk@daa9b2d5 (dotnet#7192)
      [Mono.Android] Optional NTLMv2 support in AndroidMessageHandler (dotnet#6999)
      Bump to dotnet/installer@53587f9 7.0.100-rc.1.22374.1 (dotnet#7198)
    grendello committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    e6101c1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    657aff0 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. Configuration menu
    Copy the full SHA
    e8d3026 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2022

  1. Merge branch 'main' into mm-codegen

    * main:
      Bump to dotnet/java-interop@a5756ca8. (dotnet#7226)
      Bump `$(AndroidNet6Version)` to 32.0.447 (dotnet#7224)
    grendello committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    4d7ec30 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      Bump to dotnet/java-interop@a5756ca8. (dotnet#7226)
      Bump `$(AndroidNet6Version)` to 32.0.447 (dotnet#7224)
    grendello committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    39068d3 View commit details
    Browse the repository at this point in the history
  3. Generate correct native names and process overloads properly

    Also, deal with duplicate native symbol names
    grendello committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    82ece43 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    acd5bc8 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'main' into mm-codegen

    * main:
      [Mono.Android] fix crash on startup with EnableLLVM (dotnet#7188)
      [ci] Add Android Designer test template (dotnet#7227)
    grendello committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    9a476d8 View commit details
    Browse the repository at this point in the history
  6. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      [Mono.Android] fix crash on startup with EnableLLVM (dotnet#7188)
      [ci] Add Android Designer test template (dotnet#7227)
    grendello committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    8d2503f View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2022

  1. Merge branch 'main' into mm-codegen

    * main:
      [Localization] Import translated resx files (dotnet#7190)
    grendello committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    f913d24 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      [Localization] Import translated resx files (dotnet#7190)
    grendello committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    52c99db View commit details
    Browse the repository at this point in the history
  3. Trying to get the MAUI sample app running

    Hit a snag with one of the callbacks using non-blittable
    arguments (`bool` in this case), but it seems to be the last (famous
    last words) obstacle preventing the app from starting.
    
    A handful of hacks are needed ATM, too.
    grendello committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    e5358f4 View commit details
    Browse the repository at this point in the history
  4. Merge branch 'main' into mm-codegen

    * main:
      [build] update to latest JDKs (dotnet#7236)
    grendello committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    2f15f67 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      [build] update to latest JDKs (dotnet#7236)
    grendello committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    21c066f View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2022

  1. MAUI app ran with marshal methods for the first time

    Unfortunately, 133 out of 182 methods are still registered dynamically,
    the reason being that they either return `bool` or take it as one of
    their parameters. `bool` isn't a blittable type and thus such methods
    cannot be used with `[UnregisteredCallersOnly]`. To move farther, we
    need to modify the generator to stop generating native callbacks (just
    them) with `bool`.
    grendello committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    3c8901a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7dc40fc View commit details
    Browse the repository at this point in the history
  3. Merge branch 'main' into mm-codegen

    * main:
      Bump to dotnet/installer@8f63969 7.0.100-rc.1.22407.1 (dotnet#7217)
    grendello committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    2b2ce55 View commit details
    Browse the repository at this point in the history
  4. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      Bump to dotnet/installer@8f63969 7.0.100-rc.1.22407.1 (dotnet#7217)
    grendello committed Aug 8, 2022
    Configuration menu
    Copy the full SHA
    9c38c68 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2022

  1. Merge branch 'main' into mm-codegen

    * main:
      [ci] Upload test assemblies after signing (dotnet#7241)
    grendello committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    85f2cfa View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      [ci] Upload test assemblies after signing (dotnet#7241)
    grendello committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    024b5f5 View commit details
    Browse the repository at this point in the history
  3. We need to generate wrappers for methods with non-blittable params

    Beginnings of the generator
    grendello committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    4d92570 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2022

  1. Configuration menu
    Copy the full SHA
    20a2f00 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2022

  1. Non-blittable marshal method wrappers implemented

    The app still crashes for some reason (appears something is not
    decorated with the `[UnmanagedCallersOnly]` attribute) but the IL code
    generation is complete and the app doesn't report any native/runtime
    linking errors.
    
    TBC next week
    grendello committed Aug 12, 2022
    Configuration menu
    Copy the full SHA
    2d3c043 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2022

  1. Merge branch 'main' into mm-codegen

    * main:
      Bump to xamarin/monodroid@210073e1 (dotnet#7272)
      [OneLoc] Localize Microsoft.Android.Templates (dotnet#7248)
      [README] Add links to XA 13.0 release installers (dotnet#7251)
      Bump to dotnet/installer@716bd17 7.0.100-rc.1.22409.23 (dotnet#7247)
      Bump manifest-merger from 30.2.1 to 30.2.2 (dotnet#7238)
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    071e9d3 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      Bump to xamarin/monodroid@210073e1 (dotnet#7272)
      [OneLoc] Localize Microsoft.Android.Templates (dotnet#7248)
      [README] Add links to XA 13.0 release installers (dotnet#7251)
      Bump to dotnet/installer@716bd17 7.0.100-rc.1.22409.23 (dotnet#7247)
      Bump manifest-merger from 30.2.1 to 30.2.2 (dotnet#7238)
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    710242e View commit details
    Browse the repository at this point in the history
  3. Fix non-blittable wrapper method generation

    MAUI hello world works now
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    af91895 View commit details
    Browse the repository at this point in the history
  4. Merge branch 'main' into mm-codegen

    * main:
      [ci] disable `<XamarinTelemetry/>` task (dotnet#7275)
      [Xamarin.Android.Build.Tasks] Use marshal-ilgen component (dotnet#7260)
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    673836c View commit details
    Browse the repository at this point in the history
  5. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      [ci] disable `<XamarinTelemetry/>` task (dotnet#7275)
      [Xamarin.Android.Build.Tasks] Use marshal-ilgen component (dotnet#7260)
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    ca2f92d View commit details
    Browse the repository at this point in the history
  6. Merge branch 'main' into mm-codegen

    * main:
      [xabuild] update binding redirects for MSBuild 17.3 (dotnet#7273)
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    10e63e2 View commit details
    Browse the repository at this point in the history
  7. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      [xabuild] update binding redirects for MSBuild 17.3 (dotnet#7273)
    grendello committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    238417e View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2022

  1. Disable some debug CWLs

    grendello committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    72c4b60 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      Disable some debug CWLs
    grendello committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    eb94adc View commit details
    Browse the repository at this point in the history
  3. Add some debugging info to function pointer lookups

    We can now report names of the mathods and classes that we failed
    to look up.
    grendello committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    a8471cd View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cfaabf2 View commit details
    Browse the repository at this point in the history
  5. Merge branch 'main' into mm-codegen

    * main:
      Bump to dotnet/installer@f3da421 7.0.100-rc.2.22417.1 (dotnet#7255)
    grendello committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    8ca61bb View commit details
    Browse the repository at this point in the history
  6. Merge branch 'mm-codegen' into mm-runtime

    * mm-codegen:
      Escape double quotes in function attribute value
      Bump to dotnet/installer@f3da421 7.0.100-rc.2.22417.1 (dotnet#7255)
    grendello committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    a473f27 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2022

  1. Configuration menu
    Copy the full SHA
    d99b7e3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cfa1b30 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'main' into mm-runtime

    * main:
      [Xamarin.Android.Build.Tasks, monodroid] LLVM-IR Generator (dotnet#7163)
      $(AndroidPackVersionSuffix)=rc.2; net7 is 33.0.0.rc.2 (dotnet#7283)
      [ci] Add EXE files to windows-toolchain-pdb artifact (dotnet#7279)
    grendello committed Aug 19, 2022
    Configuration menu
    Copy the full SHA
    9c9612d View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    334619b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c9332a8 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2022

  1. Remove workarounds for methods with non-blittable types

    These are now handled by the wrapper generator
    grendello committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    e518a4e View commit details
    Browse the repository at this point in the history
  2. Update apkdesc files

    grendello committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    6ca1245 View commit details
    Browse the repository at this point in the history
  3. Fix a failing test

    grendello committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    63bf193 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      Bump r8 from 3.3.28 to 3.3.75 (dotnet#7292)
      [Xamarin.Android.Build.Tasks] Pass `hybrid` to AOT, if enabled (dotnet#7263)
      Bump to lz4/lz4@5ff83968 [v1.9.4] (dotnet#7262)
      [Mono.Android] add "built-in" delegate for MAUI+non-Shell (dotnet#7267)
    grendello committed Aug 23, 2022
    Configuration menu
    Copy the full SHA
    c516792 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c8a2d9f View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      [xaprepare] Generate SourceLink.json (dotnet#7298)
    grendello committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    0aa3e95 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'main' into mm-runtime

    * main:
      Bump to dotnet/installer@84d26a0 7.0.100-rc.2.22419.24 (dotnet#7294)
      Bump to xamarin/java.interop/main@e31d9c62 (dotnet#7266)
    grendello committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    49895b8 View commit details
    Browse the repository at this point in the history
  3. Remove JI workaround

    grendello committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    92fa671 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      [tests] disable `$(_FastDeploymentDiagnosticLogging)` in PerformanceTest (dotnet#7307)
    grendello committed Aug 26, 2022
    Configuration menu
    Copy the full SHA
    6b116bf View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      [ci] Split up signing for files to be notarized (dotnet#7321)
      [ci] Add support to net7.0 for multi-targeting in VS (dotnet#7311)
      [ci] Skip classic tests on .NET release branches (dotnet#7312)
      [tests] Use `$(AndroidSdkPlatformToolsVersion)`=33.0.3 (dotnet#7320)
      Bump to xamarin/java.interop/main@4f813cb2 (dotnet#7308)
      Bump to mono/mono.posix@d8994ca, dotnet/android-libzipsharp@98e9173 (dotnet#7309)
    grendello committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    e1e59f0 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      Bump to dotnet/installer@2d1a4de 7.0.100-rc.2.22426.5 (dotnet#7314)
      [FabricBot] Apply 'needs-triage' label to new issues. (dotnet#7327)
      [Mono.Android] .NET 6+ & GetHttpMessageHandler & null, oh my! (dotnet#7214)
    grendello committed Sep 2, 2022
    Configuration menu
    Copy the full SHA
    df04d47 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      [FabricBot] Add `possibly-stale` GitHub label for closing old issues (dotnet#7343)
      Don't use Dictionary<K,V> to avoid duplicate value exception (dotnet#7340)
      [One .NET] fix `dotnet run -c Release` (dotnet#7341)
    grendello committed Sep 6, 2022
    Configuration menu
    Copy the full SHA
    d3d9007 View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2022

  1. Merge branch 'main' into mm-runtime

    * main:
      Bump to dotnet/installer@330dee3 (dotnet#7334)
      [xaprepare] CGManifest.json conforms to JSON schema (dotnet#7342)
    grendello committed Sep 7, 2022
    Configuration menu
    Copy the full SHA
    b459e4a View commit details
    Browse the repository at this point in the history