Rejit not working under .NET Core

[weltkante]

I have an instrumentation method that requires rejit during runtime to rewrite specified target methods (inject instrumentation calls) and it works under Desktop Framework, but when running under .NET Core (tested with 7 and 8rc) the rejit call doesn't seem to do anything. Unlike when running under Desktop Framework I'm not given the chance to rewrite the target IL, ShouldInstrumentMethod is never called, .NET Core simply seems to not react on the rejit request (but it doesn't generate an error either).

If the method to be instrumented hasn't been called yet (and thus no rejit is required) my instrumentation works as expected, only the case where the subject method already got called before I do the rejit request doesn't work. I've tried making sure inlining is not a problem by annotating the method accordingly and in Desktop Framework I'm getting ShouldInstrumentMethod with isRejit=true parameter as expected.

Do you have any suggestions how to proceed diagnosing the issue? There is very little information and examples about rejit available. The NaglerInstrumentationMethod just calls it during OnModuleLoaded (which seems a weird place to test rejit functionality since at that point nothing has been jitted yet?)

If you don't have any suggestions or references for me to look at I can start building a standalone repro to post, but that will take a while, so I'm hoping there is more I can check on my own.

[delmyers]

I have a couple of follow-up questions.

1. Are you getting any errors back from the API for requesting a rejit?
2. Do you know what thread you are on when you make the rejit request? Technically speaking, the CLR documentation says that you should ensure that the rejit request is not called on a CLR thread, because it can cause issues. You should create a thread within your app, and make the rejit request there.

A standalone repro would be nice. There might be a new bug in the CLR because rejit is relatively low use.

[weltkante]

Are you getting any errors back from the API for requesting a rejit?

nope, the rejit is accepted the same in both Desktop Framework and .NET Core (no error HRESULT returned from the request)

Do you know what thread you are on when you make the rejit request?

I'm following the docs here and do the request from a dedicated (non-dotnet) thread

A standalone repro would be nice. There might be a new bug in the CLR because rejit is relatively low use.

If nothing else comes to mind I'll prepare one for next week, thanks for taking the time

[weltkante]

Uploaded a repro to https://github.com/weltkante/RejitTest

The solution should (hopefully) be fully configured to "just work", CLRIE is pulled in as a submodule. I'm using the current VS 2022 preview for testing and the setup is configured to build and launch both x86/x64 and net472/net7/net8 builds in a mixed mode debugger able to set breakpoints in the C++ code. Environment variables are set in the launchsettings.json and target the build output locations of VS so should not require copying files around.

The test code is very simplified, I created only a very minimalistic instrumentation method counting calls, to demonstrate that rejit works in net472 but (apparently) not in netcore. This reflects our usecase of using the profiling API for unit testing where instrumentation is per unit test and a fully preconfigured setup isn't possible.

The test scenario should be self-documenting in the main function and console output. We count invocations of a test method, which itself is called once before and after instrumentation (including rejit) is requested. It only works when running on Desktop Framework as described previously in the issue. Commenting out the first call, so the method is not yet seen by the JIT before instrumentation is requested (and no rejit required), makes it work on all frameworks.

[delmyers]

Thanks, @weltkante. I ran through the test today, and I can't see anything wrong. Even stranger is that things seem to function as expected when a debugger is attached. This indicates to me that there is some sort of timing issue in the CLR that is preventing rejit from running.

@davmason would you be able to consult on this issue? It seems that rejit isn't working as expected. I've tried using our current code that attempts to track inline sites for rejit, as well as RequestReJITWithInliners, and neither seems to work for .NET 7 or 8 unless the debugger is attached. I don't know what additional configuration the clr might be inferring because of the debugger being attached, but that does seem to effect the outcome.

[delmyers]

@weltkante Just as a quick update, I have the scenario a try just from a regular command prompt, after setting all of the correct environment variables, and things worked as expected. The only time it doesn't seem to work is when running inside VS without the debugger attached.

[delmyers]

Hmm... it also works with VS launch if running in "Release" instead of "Debug". This is getting more curious.

[delmyers]

OK. I think I figured it out. The problem is probably that you have "Hot Reload" enabled when starting without debugging. Hot reload is incompatible with .NET rejit. I got your scenario to work by disabling hot reload in the settings:

[weltkante]

Huh, ok, that did indeed help. I had both checkboxes checked by default so hot reload apparently was on both when running with and without debugger. Disabling it indeed fixes the problem, thank you very much!

Very weird that VS modifies the process when I explicitly ask it to run it without debugging - and that the runtime silently ignores rejit rather than failing the profiler feature request and/or the rejit quest. Is there anything to follow up and improve on? Otherwise I think this can be closed. Thanks again.

[delmyers]

It is a relatively new feature in Visual Studio. It is to allow people who are just doing testing and development to see the effects of their changes without the overhead of the debugger or having to hit a breakpoint to make changes.

The details are here: https://learn.microsoft.com/en-us/visualstudio/debugger/hot-reload?view=vs-2022

Of note, you can edit your launch.json to disable hot reload:

{
  "profiles": {
    "Console": {
      "commandName": "Project",
      "hotReloadEnabled": false
    }
  }
}

I'm unsure if there is a way that we can detect that hot reload is enabled while the application is running. There might be an environment variable or something. I can look into it. If yes, then we might be able to return an error code with the rejit request. I would prefer it if the CLR did that itself, though. Even better would be if hot reload was made compatible with rejit. That probably wouldn't be something that would become available until .NET 9 or 10, though.

[delmyers]

@weltkante you could consider filing a feature request with the dotnet team: https://github.com/dotnet/runtime/issues

[weltkante]

Thanks, I agree this is better handled at the runtime than here.

[weltkante]

tracked under dotnet/runtime#91963