New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Profiling-API callback for AssemblyBindFailure's #3894

Open
discostu105 opened this Issue Mar 23, 2016 · 20 comments

Comments

Projects
None yet
8 participants
@discostu105
Contributor

discostu105 commented Mar 23, 2016

Here is my use-case:

  • I'm trying to write a Profiler for the CoreCLR (which is hooked into the process via Profiling API (CORECLR_ENABLE_PROFILING, ...)
  • It's intercepting ModuleLoad and JitCompilationStarted callbacks to modify Module Metadata on the fly and inject IL code into certain methods.
  • The injected IL instructions call a managed helper method, which is defined in a separate assembly (MyProfiler.dll, MyProfiler.MethodEnter())
  • In order to inject that IL instruction, I need to add a TypeRef/MemberRef for MyProfiler.MethodEnter() and an AssemblyRef to MyProfiler.dll as resolution scope. The AssemblyRef would point to MyProfiler with a certain version, hashcode, ect.
  • Since the AssemblyRef to MyProfiler.dll is injected at runtime, that assembly is not referenced by the application at build-time. So, there is nothing about it in [AssemblyName].deps

My question is: How can I make the CoreCLR resolve that AssemblyRef? I already figured, Assemblies are searched in the application directory and in the "Trusted Platform Assemblies". If I understood that right, these are passed in by the corehost, and usually they come from the global nuget package cache.

So, I see three possible solutions for this:
(1) add MyProfiler.dll to the nuget package cache on the machine
(2) somehow add MyProfiler.dll to the TPA list (is that possible from within a profiler?)
(3) Add a Profiling-API callback like AssemblyBindFailure and allow the profiler to specify a path to the dll. The CLR would then try to bind that assembly once more with the given path.

I'm not very much in favor of (1), because it would involve some "installer-step" for my profiler to get it to work. I thought (3) is a neat idea that would solve my problem elegantly. What do you think about it?

Or is there a much simpler solution to my problem?

@jkotas

This comment has been minimized.

Show comment
Hide comment
@jkotas

jkotas Mar 24, 2016

Member

@noahfalk This must be a common problem, even for profilers on full .NET Framework. How do profilers solve it typically?

Member

jkotas commented Mar 24, 2016

@noahfalk This must be a common problem, even for profilers on full .NET Framework. How do profilers solve it typically?

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Mar 24, 2016

Member

Just a heads up, profiling on .Net Core is not a fully fleshed out scenario at this point. Some of our community members have ported a subset of the APIs and I am not blocking anyone from using them, but its very much a 'try at your own risk' area right now. Check out #445 to see the ongoing discussion. I'm happy to discuss issues and provide what advice I can - just know that pre-canned solutions may not exist yet.

In regards to how this type of problem was solved on desktop, there hasn't been a one-size fits all solution yet. A common approach was that profilers would inject their assemblies into the GAC. For UWP apps I think profilers were doing complicated things with file system permissions to inject their assemblies into the folders that the app's AppContainer ran from. Effectively the requirements of each host/app-model was considered and solved individually.

For solutions in the .Net Core space (just off the top of my head, further thought and investigation could easily change this)...
Certainly one option is to continue doing per-host solutions such as your (1) and some variants of (2), but I agree that these integration points are often overly complex, don't allow profilers to easily move between app-models, and usually require installation dependencies/permissions you'd prefer to avoid if possible. The other category of solutions that feel more appealing are host agnostic ones where the runtime only coordinates with the profiler. (3) is one way such a system could work but I am sure there are others such as providing specific assemblies in API up-front, or maybe some way to integrate with the AssemblyLoadContext construct. If we were going to do one of these I think the next step would be for someone to do a bit more investigation about the pros/cons of the choices and write up proposal that is a little more concrete about what exactly is the API and how does it interact with the runtime. I'm fine taking it as a feature suggestion for our backlog, or of course if you wanted to push ahead you are welcome to investigate more now.

One other thing I should mention is that there is a technique to workaround the assembly resolution problem rather than tackling it head on. Its not trivial and probably not the ideal long term solution for the platform, but I expect it will work without requiring additional runtime feature work. Rather than adding a reference to a new assembly that contains your profiler code, you inject all the profiler code/metadata into some assembly that already exists, such as System.Runtime. Then you use the pre-existing AssemblyRef for System.Runtime as the scope for all your profiler TypeRefs and MemberRefs. In some earlier versions of .Net there were some limitations that prevented adding new types to assemblies (profilers would add new methods to existing types as a workaround), but I think those issues were resolved around .Net 4.6 timeframe and the updates should have flowed into coreclr too. In the limit you can include a managed dll such as MyProfiler.dll with your profiler deployment but at runtime instead of referencing the assembly directly you read the contents of the assembly with a metadata reader and then inject semantically identical contents into System.Runtime. This allows you to develop your managed profiler assembly with standard tools as you probably do now without requiring the runtime to ever reference the assembly at runtime.

Member

noahfalk commented Mar 24, 2016

Just a heads up, profiling on .Net Core is not a fully fleshed out scenario at this point. Some of our community members have ported a subset of the APIs and I am not blocking anyone from using them, but its very much a 'try at your own risk' area right now. Check out #445 to see the ongoing discussion. I'm happy to discuss issues and provide what advice I can - just know that pre-canned solutions may not exist yet.

In regards to how this type of problem was solved on desktop, there hasn't been a one-size fits all solution yet. A common approach was that profilers would inject their assemblies into the GAC. For UWP apps I think profilers were doing complicated things with file system permissions to inject their assemblies into the folders that the app's AppContainer ran from. Effectively the requirements of each host/app-model was considered and solved individually.

For solutions in the .Net Core space (just off the top of my head, further thought and investigation could easily change this)...
Certainly one option is to continue doing per-host solutions such as your (1) and some variants of (2), but I agree that these integration points are often overly complex, don't allow profilers to easily move between app-models, and usually require installation dependencies/permissions you'd prefer to avoid if possible. The other category of solutions that feel more appealing are host agnostic ones where the runtime only coordinates with the profiler. (3) is one way such a system could work but I am sure there are others such as providing specific assemblies in API up-front, or maybe some way to integrate with the AssemblyLoadContext construct. If we were going to do one of these I think the next step would be for someone to do a bit more investigation about the pros/cons of the choices and write up proposal that is a little more concrete about what exactly is the API and how does it interact with the runtime. I'm fine taking it as a feature suggestion for our backlog, or of course if you wanted to push ahead you are welcome to investigate more now.

One other thing I should mention is that there is a technique to workaround the assembly resolution problem rather than tackling it head on. Its not trivial and probably not the ideal long term solution for the platform, but I expect it will work without requiring additional runtime feature work. Rather than adding a reference to a new assembly that contains your profiler code, you inject all the profiler code/metadata into some assembly that already exists, such as System.Runtime. Then you use the pre-existing AssemblyRef for System.Runtime as the scope for all your profiler TypeRefs and MemberRefs. In some earlier versions of .Net there were some limitations that prevented adding new types to assemblies (profilers would add new methods to existing types as a workaround), but I think those issues were resolved around .Net 4.6 timeframe and the updates should have flowed into coreclr too. In the limit you can include a managed dll such as MyProfiler.dll with your profiler deployment but at runtime instead of referencing the assembly directly you read the contents of the assembly with a metadata reader and then inject semantically identical contents into System.Runtime. This allows you to develop your managed profiler assembly with standard tools as you probably do now without requiring the runtime to ever reference the assembly at runtime.

@sergiy-k sergiy-k added this to the 1.0.0-rtm milestone Mar 24, 2016

@noahfalk noahfalk modified the milestones: Future, 1.0.0-rtm Mar 24, 2016

@discostu105

This comment has been minimized.

Show comment
Hide comment
@discostu105

discostu105 Mar 24, 2016

Contributor

Noah, thanks for your detailed answer!

The "workaround" you described, is what I have already prototyped. It basically works as you described (I'm injecting into mscorlib directly though). However, as soon as my profiler-code itself has other dependencies, I start to run into troubles and have some uncertainties. Initially I thought, loading MyProfiler.dll as it is, would prevent some of these problems (but maybe they'd be just the same):

E.g. What if my profiler wanted to use System.Threading.Thread or System.Diagnostics.Stacktrace packages? I would have to add those AssemblyRefs to mscorlib (where my profiler-helper-methods now live in). A concrete problem I ran into so far, was that my application did reference System.Runtime 4.0, but my newly added reference to System.Threading.Thread 4.0, indirectly depended on System.Runtime 4.1. Then things went south, because AssemblyBinder::IsValidAssemblyVersion checks major.minor versions against already loaded assemblies with the same name (right?).

Another uncertainty for me is: what if I add an AssemblyRef to a package, which is not yet restored in the nuget package cache of the target system. Would it even be part of TPA if it's not in the .deps file? Wouldn't I run into the same problem again with those dependencies? Again, I would first have to install my profilers dependencies into the nuget cache, otherwise the CLR would not find them? In that case, an approach could be that the profiler ships with all packages it depends on out of the box and it could give the CLR a hint where to find those (at runtime) via proposed solutions (2) or (3).

There seems to be a way to log assembly-bindings for debugging. I tried setting COMPlus_CoreClrBinderLog=1 as env variable, but I did not see any logs. Can you give me a hint on how to enable this?

By the way: For troubleshooting so far, I have compiled coreclr myself to be able to debug and step directly through the clr-code. This is tremendously helpful for me, compared to troubleshooting with full framework CLR! OSS FTW.

Contributor

discostu105 commented Mar 24, 2016

Noah, thanks for your detailed answer!

The "workaround" you described, is what I have already prototyped. It basically works as you described (I'm injecting into mscorlib directly though). However, as soon as my profiler-code itself has other dependencies, I start to run into troubles and have some uncertainties. Initially I thought, loading MyProfiler.dll as it is, would prevent some of these problems (but maybe they'd be just the same):

E.g. What if my profiler wanted to use System.Threading.Thread or System.Diagnostics.Stacktrace packages? I would have to add those AssemblyRefs to mscorlib (where my profiler-helper-methods now live in). A concrete problem I ran into so far, was that my application did reference System.Runtime 4.0, but my newly added reference to System.Threading.Thread 4.0, indirectly depended on System.Runtime 4.1. Then things went south, because AssemblyBinder::IsValidAssemblyVersion checks major.minor versions against already loaded assemblies with the same name (right?).

Another uncertainty for me is: what if I add an AssemblyRef to a package, which is not yet restored in the nuget package cache of the target system. Would it even be part of TPA if it's not in the .deps file? Wouldn't I run into the same problem again with those dependencies? Again, I would first have to install my profilers dependencies into the nuget cache, otherwise the CLR would not find them? In that case, an approach could be that the profiler ships with all packages it depends on out of the box and it could give the CLR a hint where to find those (at runtime) via proposed solutions (2) or (3).

There seems to be a way to log assembly-bindings for debugging. I tried setting COMPlus_CoreClrBinderLog=1 as env variable, but I did not see any logs. Can you give me a hint on how to enable this?

By the way: For troubleshooting so far, I have compiled coreclr myself to be able to debug and step directly through the clr-code. This is tremendously helpful for me, compared to troubleshooting with full framework CLR! OSS FTW.

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Mar 25, 2016

Member

This is tremendously helpful for me, compared to troubleshooting with full framework CLR! OSS FTW.

I'm very glad OSS is helping you out, that's what we hoped for : )

There seems to be a way to log assembly-bindings for debugging. I tried setting COMPlus_CoreClrBinderLog=1 as env variable, but I did not see any logs. Can you give me a hint on how to enable this?

I'm not that knowledgable about debugging the assembly binder myself but maybe @sergiy-k or @jkotas have a lead?

E.g. What if my profiler wanted to use System.Threading.Thread or System.Diagnostics.Stacktrace packages? I would have to add those AssemblyRefs to mscorlib (where my profiler-helper-methods now live in).

Ideally I'd say keep your profiler's in-process dependencies as lean as you can possibly make them - which hopefully avoids the need to depend on either of those assemblies. For those particular assemblies you mentioned, you might find that the types you need are already inside of mscorlib. Those assemblies are mostly just forwarding back to mscorlib's types as I recall. This does of course bind your profiler even more tightly to a particular implementation of the platform and its quite possible things will break in the future and be non-portable to other implementations. Effectively the choice is between reducing dependencies in-proc, doing brittle unsupported things, or using integration points that may not exist yet.

A concrete problem I ran into so far, was that my application did reference System.Runtime 4.0 , but my newly added reference to System.Threading.Thread 4.0 , indirectly depended on System.Runtime 4.1 .

Yeah another challenge of your profiler having in-proc dependencies is that you often have to support using whatever version of the components are present, or declaring that your profiler simply doesn't run on apps that don't already reference compatible versions of the assemblies you want to use. I don't know what scenario your profiler is trying to address so I have no idea if that type of restriction is feasible. For many profilers I know it is not feasible and they simply avoid taking the dependencies that cause these versioning issues. In the full CLR you might have been able to use a second AppDomain but in .NET Core's one AppDomain the only way to get enough isolation to load arbitrary assembly versions is using another process with an IPC mechanism.

Another uncertainty for me is: what if I add an AssemblyRef to a package, which is not yet restored in the nuget package cache of the target system. Would it even be part of TPA if it's not in the .deps file? Wouldn't I run into the same problem again with those dependencies? Again, I would first have to install my profilers dependencies into the nuget cache, otherwise the CLR would not find them? In that case, an approach could be that the profiler ships with all packages it depends on out of the box and it could give the CLR a hint where to find those (at runtime) via proposed solutions (2) or (3).

If you can't avoid having the dependency in the first place and the app doesn't package it for you then your profiler would have to be responsible for deploying it. Getting the runtime to actually load it goes back to the original questions and suggestions. In theory you could inject those assembly contents into mscorlib as you did with your profiler dll, but I'm certain it would be completely unsupported from the various framework library authors.

If you are interested in describing a little more what your profiler aims to do and why you think it will need dependencies on various other assemblies I'm happy to think about it a bit to see if I can come up with suggestions on how to accomplish the same task without taking the in-proc dependencies.

Member

noahfalk commented Mar 25, 2016

This is tremendously helpful for me, compared to troubleshooting with full framework CLR! OSS FTW.

I'm very glad OSS is helping you out, that's what we hoped for : )

There seems to be a way to log assembly-bindings for debugging. I tried setting COMPlus_CoreClrBinderLog=1 as env variable, but I did not see any logs. Can you give me a hint on how to enable this?

I'm not that knowledgable about debugging the assembly binder myself but maybe @sergiy-k or @jkotas have a lead?

E.g. What if my profiler wanted to use System.Threading.Thread or System.Diagnostics.Stacktrace packages? I would have to add those AssemblyRefs to mscorlib (where my profiler-helper-methods now live in).

Ideally I'd say keep your profiler's in-process dependencies as lean as you can possibly make them - which hopefully avoids the need to depend on either of those assemblies. For those particular assemblies you mentioned, you might find that the types you need are already inside of mscorlib. Those assemblies are mostly just forwarding back to mscorlib's types as I recall. This does of course bind your profiler even more tightly to a particular implementation of the platform and its quite possible things will break in the future and be non-portable to other implementations. Effectively the choice is between reducing dependencies in-proc, doing brittle unsupported things, or using integration points that may not exist yet.

A concrete problem I ran into so far, was that my application did reference System.Runtime 4.0 , but my newly added reference to System.Threading.Thread 4.0 , indirectly depended on System.Runtime 4.1 .

Yeah another challenge of your profiler having in-proc dependencies is that you often have to support using whatever version of the components are present, or declaring that your profiler simply doesn't run on apps that don't already reference compatible versions of the assemblies you want to use. I don't know what scenario your profiler is trying to address so I have no idea if that type of restriction is feasible. For many profilers I know it is not feasible and they simply avoid taking the dependencies that cause these versioning issues. In the full CLR you might have been able to use a second AppDomain but in .NET Core's one AppDomain the only way to get enough isolation to load arbitrary assembly versions is using another process with an IPC mechanism.

Another uncertainty for me is: what if I add an AssemblyRef to a package, which is not yet restored in the nuget package cache of the target system. Would it even be part of TPA if it's not in the .deps file? Wouldn't I run into the same problem again with those dependencies? Again, I would first have to install my profilers dependencies into the nuget cache, otherwise the CLR would not find them? In that case, an approach could be that the profiler ships with all packages it depends on out of the box and it could give the CLR a hint where to find those (at runtime) via proposed solutions (2) or (3).

If you can't avoid having the dependency in the first place and the app doesn't package it for you then your profiler would have to be responsible for deploying it. Getting the runtime to actually load it goes back to the original questions and suggestions. In theory you could inject those assembly contents into mscorlib as you did with your profiler dll, but I'm certain it would be completely unsupported from the various framework library authors.

If you are interested in describing a little more what your profiler aims to do and why you think it will need dependencies on various other assemblies I'm happy to think about it a bit to see if I can come up with suggestions on how to accomplish the same task without taking the in-proc dependencies.

@jkotas

This comment has been minimized.

Show comment
Hide comment
@jkotas

jkotas Mar 25, 2016

Member

COMPlus_CoreClrBinderLog=1 as env variable

This log is under BINDER_DEBUG_LOG define that is not turned on by default. Note that the assembly loading in CoreCLR is very simple compare to full framework (where you really need the fusion log to understand the complex process). The coreclr binder log did not proved to be very useful for diagnostics because of it does not tell you much extra insight over the exception message.

Member

jkotas commented Mar 25, 2016

COMPlus_CoreClrBinderLog=1 as env variable

This log is under BINDER_DEBUG_LOG define that is not turned on by default. Note that the assembly loading in CoreCLR is very simple compare to full framework (where you really need the fusion log to understand the complex process). The coreclr binder log did not proved to be very useful for diagnostics because of it does not tell you much extra insight over the exception message.

@discostu105

This comment has been minimized.

Show comment
Hide comment
@discostu105

discostu105 May 17, 2017

Contributor

What Damian says in this video (https://youtu.be/356xGRWSSfQ?t=3747) is just about what might solve my problem here.

I found environement variables DOTNET_ADDITIONAL_DEPS and ASPNETCORE_HOSTINGSTARTUPASSEMBLIES being used in https://github.com/aspnet/AzureIntegration/blob/dev/src/Microsoft.AspNetCore.AzureAppServices.SiteExtension/applicationHost.xdt but could not find any documentation about them. Are there any docs on that?

In case this effectively let's me extend the assembly-search-path via environment variable, I guess this could work for my problem.

dotnet/core-setup#1597

Contributor

discostu105 commented May 17, 2017

What Damian says in this video (https://youtu.be/356xGRWSSfQ?t=3747) is just about what might solve my problem here.

I found environement variables DOTNET_ADDITIONAL_DEPS and ASPNETCORE_HOSTINGSTARTUPASSEMBLIES being used in https://github.com/aspnet/AzureIntegration/blob/dev/src/Microsoft.AspNetCore.AzureAppServices.SiteExtension/applicationHost.xdt but could not find any documentation about them. Are there any docs on that?

In case this effectively let's me extend the assembly-search-path via environment variable, I guess this could work for my problem.

dotnet/core-setup#1597

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk May 19, 2017

Member

Perusing some code, it looks like DOTNET_ADDITIONAL_DEPS has the same effect as the --additional-deps argument, but I haven't seen any docs on it.
https://github.com/dotnet/core-setup/blob/master/src/corehost/cli/fxr/fx_muxer.cpp

I did see that it appears mutually exclusive with the command line argument though which might be an issue?

Maybe a question to the folks over in that repo would get a better answer. From the CLR's perspective we load whatever the host tells us to load without knowing what policy mechanisms the host used to obtain their list.

Member

noahfalk commented May 19, 2017

Perusing some code, it looks like DOTNET_ADDITIONAL_DEPS has the same effect as the --additional-deps argument, but I haven't seen any docs on it.
https://github.com/dotnet/core-setup/blob/master/src/corehost/cli/fxr/fx_muxer.cpp

I did see that it appears mutually exclusive with the command line argument though which might be an issue?

Maybe a question to the folks over in that repo would get a better answer. From the CLR's perspective we load whatever the host tells us to load without knowing what policy mechanisms the host used to obtain their list.

@pkruk2

This comment has been minimized.

Show comment
Hide comment
@pkruk2

pkruk2 Mar 23, 2018

Hi, is there any change in this area?

I'm facing the same issue. I have a profiler which injects IL with calls to some managed methods which are defined in my assembly. The issue is that the assembly is not found during runtime.

Adding my assembly to nuget package cache didn't work - isn't it supposed to be used only when the assembly is listed as reference in .deps.json file anyway?

The DOTNET_ADDITIONAL_DEPS workaround helps, but I don't feel comfortable using it. You mentioned that it's mutually exclusive with the "--additional-deps". Also, it requires setting runtimeTarget which I'm not sure yet if I can discover reliably from profiling api.

Minimal DOTNET_ADDITIONAL_DEPS that worked for me is:

{
  "runtimeTarget": {
    "name": ".NETCoreApp,Version=v2.0"
  },
  "targets": {
    ".NETCoreApp,Version=v2.0": {
      "my_assembly/version": {
        "runtime": {
          "path_to_my_assembly": {}
        }
      }
    }
  },
  "libraries": {
    "my_assembly/version": {
      "type": "reference",
      "serviceable": false,
      "sha512": ""
    }
  }
}

So I hope there is a better way of doing it now...

Thanks for help.

pkruk2 commented Mar 23, 2018

Hi, is there any change in this area?

I'm facing the same issue. I have a profiler which injects IL with calls to some managed methods which are defined in my assembly. The issue is that the assembly is not found during runtime.

Adding my assembly to nuget package cache didn't work - isn't it supposed to be used only when the assembly is listed as reference in .deps.json file anyway?

The DOTNET_ADDITIONAL_DEPS workaround helps, but I don't feel comfortable using it. You mentioned that it's mutually exclusive with the "--additional-deps". Also, it requires setting runtimeTarget which I'm not sure yet if I can discover reliably from profiling api.

Minimal DOTNET_ADDITIONAL_DEPS that worked for me is:

{
  "runtimeTarget": {
    "name": ".NETCoreApp,Version=v2.0"
  },
  "targets": {
    ".NETCoreApp,Version=v2.0": {
      "my_assembly/version": {
        "runtime": {
          "path_to_my_assembly": {}
        }
      }
    }
  },
  "libraries": {
    "my_assembly/version": {
      "type": "reference",
      "serviceable": false,
      "sha512": ""
    }
  }
}

So I hope there is a better way of doing it now...

Thanks for help.

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Mar 23, 2018

Member

Nobody directly on the diagnostics team has made any changes at least. For some profilers I believe the priority of this issue diminished as the 2.0 framework started including more of the traditional .NET SDK surface area by default. Your request now is the first I had heard on the topic in a while.

I haven't done this myself so I can't guarantee it works or that it is a fully fleshed out idea, but I'd suspect hooking the Resolving event for AssemblyLoadContext.Default may also be a viable way to get dynamically injected dependencies to resolve.

Member

noahfalk commented Mar 23, 2018

Nobody directly on the diagnostics team has made any changes at least. For some profilers I believe the priority of this issue diminished as the 2.0 framework started including more of the traditional .NET SDK surface area by default. Your request now is the first I had heard on the topic in a while.

I haven't done this myself so I can't guarantee it works or that it is a fully fleshed out idea, but I'd suspect hooking the Resolving event for AssemblyLoadContext.Default may also be a viable way to get dynamically injected dependencies to resolve.

@pkruk2

This comment has been minimized.

Show comment
Hide comment
@pkruk2

pkruk2 Mar 26, 2018

Hi noahfalk, thanks for update.

For some profilers I believe the priority of this issue diminished as the 2.0 framework started including more of the traditional .NET SDK surface area by default.

I am not sure how this will help here?

I'd suspect hooking the Resolving event for AssemblyLoadContext.Default may also be a viable way to get dynamically injected dependencies to resolve.

I'm not sure if I understand correctly. Do you mean to do this in C# ? But this would require having my c# assembly already loaded, and we are going back to the problem with how to load an assembly.

Or do you mean to hook the Resolving event from C++ profiling code somehow?

pkruk2 commented Mar 26, 2018

Hi noahfalk, thanks for update.

For some profilers I believe the priority of this issue diminished as the 2.0 framework started including more of the traditional .NET SDK surface area by default.

I am not sure how this will help here?

I'd suspect hooking the Resolving event for AssemblyLoadContext.Default may also be a viable way to get dynamically injected dependencies to resolve.

I'm not sure if I understand correctly. Do you mean to do this in C# ? But this would require having my c# assembly already loaded, and we are going back to the problem with how to load an assembly.

Or do you mean to hook the Resolving event from C++ profiling code somehow?

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Mar 27, 2018

Member

For some profilers I believe the priority of this issue diminished as the 2.0 framework started including more of the traditional .NET SDK surface area by default.

I am not sure how this will help here?

I only mentioned it as context for why others cared in the past and then the priority diminished, I didn't intend it to be a solution to the current issue.

I'm not sure if I understand correctly. Do you mean to do this in C# ? But this would require having my c# assembly already loaded, and we are going back to the problem with how to load an assembly.

I meant using IL instrumentation to inject modified code into one of the already existing framework assemblies. Effectively anything we can write in C# into the framework source, your profiler can also write at runtime via IL instrumentation, albeit a little more tediously. My suggestion if you wanted to explore this route would be to get a copy of CoreFX source and try making the source changes manually first in C# and substitute your modified binary into the framework installation. If that works properly then you could tackle the next step of making the same modification using only the profiler IL instrumentation APIs. Does that make a bit more sense? In terms of work I think this is certainly more effort than additional deps that you are already using, but at the moment its the only other option I can think of that would work with the runtime as-is.

Ultimately I think we will need to do something in the runtime to improve the support around this scenario. If you had any interest in working on it from that angle thats another possibility?

Member

noahfalk commented Mar 27, 2018

For some profilers I believe the priority of this issue diminished as the 2.0 framework started including more of the traditional .NET SDK surface area by default.

I am not sure how this will help here?

I only mentioned it as context for why others cared in the past and then the priority diminished, I didn't intend it to be a solution to the current issue.

I'm not sure if I understand correctly. Do you mean to do this in C# ? But this would require having my c# assembly already loaded, and we are going back to the problem with how to load an assembly.

I meant using IL instrumentation to inject modified code into one of the already existing framework assemblies. Effectively anything we can write in C# into the framework source, your profiler can also write at runtime via IL instrumentation, albeit a little more tediously. My suggestion if you wanted to explore this route would be to get a copy of CoreFX source and try making the source changes manually first in C# and substitute your modified binary into the framework installation. If that works properly then you could tackle the next step of making the same modification using only the profiler IL instrumentation APIs. Does that make a bit more sense? In terms of work I think this is certainly more effort than additional deps that you are already using, but at the moment its the only other option I can think of that would work with the runtime as-is.

Ultimately I think we will need to do something in the runtime to improve the support around this scenario. If you had any interest in working on it from that angle thats another possibility?

@pkruk2

This comment has been minimized.

Show comment
Hide comment
@pkruk2

pkruk2 Mar 28, 2018

Does that make a bit more sense?

I think yes. If I understand correctly, you are suggesting injecting code into existing assembly like mscorlib or System.Runtime (one that will be always loaded/initialized by runtime). The injected code would add hook for Resolving event.

This way, when the other injected code calls method from my assembly, the Resolving hook will run and will help finding my assembly.

This is indeed more work, and I think it depends mscorlib (or similar) code, that is if the code changes, my instrumentation may fail.

Ultimately I think we will need to do something in the runtime to improve the support around this scenario. If you had any interest in working on it from that angle thats another possibility?

Improved support for this definitely would be welcome. Unfortunately, I think I won't be able to help you with this now. I'll try to use the DOTNET_ADDITIONAL_DEPS workaround and if there are problems I will revisit other options.

Thanks for the help.

pkruk2 commented Mar 28, 2018

Does that make a bit more sense?

I think yes. If I understand correctly, you are suggesting injecting code into existing assembly like mscorlib or System.Runtime (one that will be always loaded/initialized by runtime). The injected code would add hook for Resolving event.

This way, when the other injected code calls method from my assembly, the Resolving hook will run and will help finding my assembly.

This is indeed more work, and I think it depends mscorlib (or similar) code, that is if the code changes, my instrumentation may fail.

Ultimately I think we will need to do something in the runtime to improve the support around this scenario. If you had any interest in working on it from that angle thats another possibility?

Improved support for this definitely would be welcome. Unfortunately, I think I won't be able to help you with this now. I'll try to use the DOTNET_ADDITIONAL_DEPS workaround and if there are problems I will revisit other options.

Thanks for the help.

@bobuva

This comment has been minimized.

Show comment
Hide comment
@bobuva

bobuva Apr 16, 2018

@noahfalk I have a profiler that injects IL into some methods so that they can call into my managed assembly. Since my assembly is in the GAC, it works fine for .NET Framework, but when I tried it in .NET Core, the assembly could not be found. I'm not sure what you meant by

the 2.0 framework started including more of the traditional .NET SDK surface area by default

as regards this issue, but I would be willing to work with you on finding a reliable way for the runtime to provide support for this assembly loading scenario.

bobuva commented Apr 16, 2018

@noahfalk I have a profiler that injects IL into some methods so that they can call into my managed assembly. Since my assembly is in the GAC, it works fine for .NET Framework, but when I tried it in .NET Core, the assembly could not be found. I'm not sure what you meant by

the 2.0 framework started including more of the traditional .NET SDK surface area by default

as regards this issue, but I would be willing to work with you on finding a reliable way for the runtime to provide support for this assembly loading scenario.

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Apr 17, 2018

Member

I would be willing to work with you on finding a reliable way for the runtime to provide support for this assembly loading scenario.

I'm happy to collaborate for whatever time you are willing to put in : ) On my side 2.1 issues are pressing so my most immediate priorities are elsewhere, but if you were interested in doing some design and/or implemention I'm sure I could carve out some time to provide design feedback and/or code review. I don't know if that's what you had in mind. If you just wanted to discuss that's fine too.

The first step would be to put together a little spec for what would get built and how a profiler would use it. My initial thoughts would be adding a new profiler API that adds to the TPA list that was provided by the host to the runtime:

HRESULT CorHost2::CreateAppDomainWithManager(

(the tpa list is one of the properties)
I think that list ultimately gets stored in the binder and then profiler could modify it there. The profiler would place its assemblies anywhere on disk it wants, then during startup invoke the API to add the paths to TPA. Later when assembly resolution occurred the profiler's assembly would be found because it is on the TPA list. If we went ahead with that approach we'd need to flesh out the details of the API. I'm also open to entirely different suggestions about how to fill this gap.

cc @jkotas @swaroop-sridhar - just a heads up in case you see any conflicts between this and other plans in the binder space

Member

noahfalk commented Apr 17, 2018

I would be willing to work with you on finding a reliable way for the runtime to provide support for this assembly loading scenario.

I'm happy to collaborate for whatever time you are willing to put in : ) On my side 2.1 issues are pressing so my most immediate priorities are elsewhere, but if you were interested in doing some design and/or implemention I'm sure I could carve out some time to provide design feedback and/or code review. I don't know if that's what you had in mind. If you just wanted to discuss that's fine too.

The first step would be to put together a little spec for what would get built and how a profiler would use it. My initial thoughts would be adding a new profiler API that adds to the TPA list that was provided by the host to the runtime:

HRESULT CorHost2::CreateAppDomainWithManager(

(the tpa list is one of the properties)
I think that list ultimately gets stored in the binder and then profiler could modify it there. The profiler would place its assemblies anywhere on disk it wants, then during startup invoke the API to add the paths to TPA. Later when assembly resolution occurred the profiler's assembly would be found because it is on the TPA list. If we went ahead with that approach we'd need to flesh out the details of the API. I'm also open to entirely different suggestions about how to fill this gap.

cc @jkotas @swaroop-sridhar - just a heads up in case you see any conflicts between this and other plans in the binder space

@bobuva

This comment has been minimized.

Show comment
Hide comment
@bobuva

bobuva Apr 17, 2018

@noahfalk I'm not that familiar with the code base but I'd like to learn more about it. Like you, I don't have a lot of cycles for this. I work for a startup and am very busy.

What would I have to do to get more familiar with the code base in this area? I'm willing to learn it but don't feel comfortable writing up a design spec for it.

By the way, I'll be in Seattle for the build conference and would be happy to meet in person to discuss more. I'm in Portland, OR, and may drive there so I would also be able to visit in Redmond.

bobuva commented Apr 17, 2018

@noahfalk I'm not that familiar with the code base but I'd like to learn more about it. Like you, I don't have a lot of cycles for this. I work for a startup and am very busy.

What would I have to do to get more familiar with the code base in this area? I'm willing to learn it but don't feel comfortable writing up a design spec for it.

By the way, I'll be in Seattle for the build conference and would be happy to meet in person to discuss more. I'm in Portland, OR, and may drive there so I would also be able to visit in Redmond.

@bobuva

This comment has been minimized.

Show comment
Hide comment
@bobuva

bobuva commented Apr 17, 2018

I found some specs in https://github.com/dotnet/cli/tree/master/Documentation/specs which will help me.

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Apr 17, 2018

Member

If you are around I'm happy to meet up. Just let me know which days/times are good. My email is noahfalk at Microsoft.com if you want to coordinate there.

If you just want to chat a bit on a high level approach and leave it for someone else to build at a later time that's fine. If you do decide you'd like to push further here are some resources to start learning more:

Docs:
https://github.com/dotnet/coreclr/tree/master/Documentation/botr
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/profiling.md
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/profilability.md

Code:
https://github.com/dotnet/coreclr/blob/master/src/vm/proftoeeinterfaceimpl.h - the implementation of the ICorProfilerInfo* interfaces.

return m_pTrustedPlatformAssemblyMap;
- where the TPA list is stored
SimpleNameToFileNameMap * tpaMap = GetAppContext()->GetTpaList();
- example code path showing use of the TPA list to resolve an assembly
In general code in the src/binder subdirectory is responsible for 'binding' - the process of resolving a reference to an assembly into a particular file on disk that should be loaded.

A good way to quickly get your hands dirty is to build coreclr and then run it under a native debugger with breakpoints set at some of the locations I mentioned. This will allow you step through and explore the code working in action. I myself haven't spent much time in the binder portion of the runtime code so I'd have to shore up my knowledge as well if we forge ahead.

Member

noahfalk commented Apr 17, 2018

If you are around I'm happy to meet up. Just let me know which days/times are good. My email is noahfalk at Microsoft.com if you want to coordinate there.

If you just want to chat a bit on a high level approach and leave it for someone else to build at a later time that's fine. If you do decide you'd like to push further here are some resources to start learning more:

Docs:
https://github.com/dotnet/coreclr/tree/master/Documentation/botr
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/profiling.md
https://github.com/dotnet/coreclr/blob/master/Documentation/botr/profilability.md

Code:
https://github.com/dotnet/coreclr/blob/master/src/vm/proftoeeinterfaceimpl.h - the implementation of the ICorProfilerInfo* interfaces.

return m_pTrustedPlatformAssemblyMap;
- where the TPA list is stored
SimpleNameToFileNameMap * tpaMap = GetAppContext()->GetTpaList();
- example code path showing use of the TPA list to resolve an assembly
In general code in the src/binder subdirectory is responsible for 'binding' - the process of resolving a reference to an assembly into a particular file on disk that should be loaded.

A good way to quickly get your hands dirty is to build coreclr and then run it under a native debugger with breakpoints set at some of the locations I mentioned. This will allow you step through and explore the code working in action. I myself haven't spent much time in the binder portion of the runtime code so I'd have to shore up my knowledge as well if we forge ahead.

@bobuva

This comment has been minimized.

Show comment
Hide comment
@bobuva

bobuva Apr 17, 2018

This looks great, thanks @noahfalk. I'll try to spend some time looking at this before I head up to //build and as we get closer to that time, I'll contact you via email if I'm ready to discuss. Love that this is all OSS now!

bobuva commented Apr 17, 2018

This looks great, thanks @noahfalk. I'll try to spend some time looking at this before I head up to //build and as we get closer to that time, I'll contact you via email if I'm ready to discuss. Love that this is all OSS now!

@sbomer

This comment has been minimized.

Show comment
Hide comment
@sbomer

sbomer Sep 13, 2018

Member

@noahfalk the startup hook makes it possible to inject code that alters assembly loading behavior. Does it look suitable for the profiler use case?
/cc @jeffschwMSFT

Member

sbomer commented Sep 13, 2018

@noahfalk the startup hook makes it possible to inject code that alters assembly loading behavior. Does it look suitable for the profiler use case?
/cc @jeffschwMSFT

@noahfalk

This comment has been minimized.

Show comment
Hide comment
@noahfalk

noahfalk Sep 13, 2018

Member

Yeah, at least for profilers that are present at startup which the main scenario today.

There are also profilers that attach to processes after they've started, but resolving assembly dependencies is one of several issues that would have to be resolved to support instrumentation on attach.

Member

noahfalk commented Sep 13, 2018

Yeah, at least for profilers that are present at startup which the main scenario today.

There are also profilers that attach to processes after they've started, but resolving assembly dependencies is one of several issues that would have to be resolved to support instrumentation on attach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment