CodeCom Evaluator gets slower after many LoadCode iterations

[ghost]

If the CodeCom Evaluator is used to compile N different scripts OR the CodeCom Evaluator is used with Caching disabled,
LoadCode becomes slower each iteration.

This can be seen from this xUnit test, created in a clone of the cs-script repo.

The preconditions for this behavior to occur are:

• EvaluatorConfig.Access = EvaluatorAccess.AlwaysCreate
• LoadCode is used, not LoadCodeRemotely to load the assembly in a separate AppDomain

The events leading up to this behavior are:

• When the CodeComEvaluator is cloned, all assemblies in the current AppDomain are referenced.
  (happens in the Reset(true) call in the CodeComEvaluator ctor)
• Because all assemblies up to iteration N-1 are already in the current AppDomain,
  all of these assemblies will be referenced on iteration N.
• The eventual call to Microsoft.CSharp.CSharpCodeGenerator.CompileAssemblyFromFileBatch,
  will have all of these assemblies in compilerParams.ReferencedAssemblies, leading to slower
  compilation

This makes it hard to use CSScript.Evaluator.LoadCode in a server environment.
Is it "by design" that all previously dynamically compiled assemblies are referenced?

Are there any work-arounds for this behavior?
(other than using LoadCodeRemotely instead of LoadCode)
(other than using EvaluatorAccess.Singleton instead of EvaluatorAccess.AlwaysCreate)

Thanks for your time,
Koen

[oleg-shilo]

It is indeed by design. Well kind of :)

The app domain assemblies are referenced by default and it is intended behavior. However inability to change this behavior is rather a mistake.
The old native CodeDom API has a dedicated config value that controls if AppDomain assemblies should be referenced by default but evaluators do not have an equivalent.

The following is a work around the problem.

var script = CSScript.CodeDomEvaluator
                     .Reset(false)
                     .LoadCode(...)

It should work but ideally there should be a config value for that. I just published on NuGet a hot fix release (v3.13.2.0) that has a new EvaluatorConfig member to address the problem directly:

CSScript.EvaluatorConfig.RefernceDomainAsemblies = false;

[ghost]

Thanks, works like a charm.
Note that you still have to set CSScript.ShareHostRefAssemblies = false;
Otherwise CSExecutor.AggregateReferencedAssemblies() would still ref all domain assemblies doing AppDomain.CurrentDomain.GetAssemblies().

One thing I noticed though: when I set this flag to false, I am no longer able to debug the dynamically loaded assembly.

Before, I could just open the generated C# file from the "dynamic" folder, and put a break point in it.
Now, when I open the file, the breakpoint gets loaded by Visual Studio, but the debugger never breaks.

Any idea why this happens?
Thanks again,
Koen

[oleg-shilo]

There are a usual suspects for debugging difficulties:

1. Debugee is elevated but the debugger is not. Most likely it's not your case as you would note that.
2. Debugger misses Debug API 'stop request'. This is usually beyond user control. In some cases it can be caused/affected by the Debugee assemblies already loaded into Debugger.
3. Debugger loads dbg symbols that do not correspond the Debuggee image

#2 and #3 Can be addressed by placing an explicit Debug.Asser(false) in the code of your interest. If you see the assert dialog and cannot attach then it's #3 who is upsetting the debugger. If you can, well... then you have your work around :)

[ghost]

Thanks, I was able to work around it.

Off-topic, a question about loading in the current AppDomain:

I am aware of the AppDomain load/unload issues and the work-around with remoting.
Since I do not want to go for the remoting option, generated assemblies are loaded into my current AppDomain.
(note that this is in a server-side environment using CodeDom; the number of generated assemblies will be in the order of 1000s).

• What are the downsides with stuffing the AppDomain with abandoned assemblies?
  I have done some profiling and the memory-overhead seems quite ok (~0.5 KB / assembly)
  Are there any other downsides?
• When running with the Mono compiler the console output prints:
  Loaded 'eval-0', Unloaded 'eval-0'. What does the "Unloaded" mean here?

Koen

[oleg-shilo]

Hi Koen,

It's actually great that you are aware about the implications of dynamic assembly loading. I tried to make it very clear in the CS-Script documentation that the implications do exist and any work around does come with the cost. To my surprise two major C# scripting alternatives (Mono and Roslyn) resorted to the blunt ignoring of the problem.

Anyway, my view on the matter was always straightforward: use local loading if you can afford it and remoting if you have to. If you don't have permanently growing footprint of loaded scripts/assemblies then probably local loading is the most practical option. The only downside apart from the memory consumption can be potential assembly probing challenges with such a wast amount of the loaded assemblies but these challenges can only become real if your scripts reference each other. If it's not your case then provably you are OK.

As for Mono compiler 'eval-0' output it's completely out of CS-Script control. These messages are from the Mono engine and I don't (well no longer) believe "Unloaded 'eval-0'" means that some dynamically loaded assembly was unloaded. You see, by endless experiments with Mono I found the way of compiling/loading the constantly changing script without increasing the memory footprint. It looks like Mono guys did some magic under the hood. This is what became CS-Script integration with Mono (Mono Evaluator). The various scenarios and their effect on memory are presented in the "/samples/Hosting/Legacy Samples/CompilerAsService/MemoryManagement.cs" sample. But... unfortunately I received user reports indicating that Mono "magic" not always work and the AppDomain the assemblies can get stuck in memory even with Mono.

Cheers,
Oleg

[oleg-shilo]

Closed as Fixed