Question: What triggers code generation?

[Kritner]

I was working on a series of blog posts to try to help (me or others) understand Orleans a bit better - in writing one of the posts I came across what I feel is an issue regarding code generation. If it is not an issue, I'd like to try to understand why one commit works, and another doesn't.

Here was my starting commit where code generation was working:
https://github.com/kritner-blogs/OrleansGettingStarted/commits/2ebd478cba2f5a6a6c679b8b6e6c56caf5518f11

The commit where code generation stopped working:
https://github.com/kritner-blogs/OrleansGettingStarted/commits/fe51806c602273afcd29ae3b126e9b964ad21659

and a comparison between the two:
Kritner-Blogs/OrleansGettingStarted@2ebd478...fe51806

When running from the first commit:

When running from the second commit:

I was able to get around this issue by having my grains and grain interfaces implement an empty interface and registering that under the respective builders, which is ok I suppose - I'm just curious if anyone can tell me why that is needed now. "Now" being in the second commit's example, but not the first.

It can be found in both commit's code, but i'm referencing:

<PackageReference Include="Microsoft.Orleans.OrleansCodeGenerator.Build" Version="2.1.0" />

In both the Grain interface and Grain Impl projects, for both the working and not working commits listed above.

[sergeybykov]

Looks like this has nothing to do with code generation. In the diff I see that you removed the only parts.AddApplicationPart() line you had. This means that the silo does not load any application grain classes, and that's why the call is failing.

Some historical background, unless you are well aware of it.

Prior to 2.0, we automatically scanned all assemblies in the silo startup folder trying to find all grain classes and interfaces there. While that made silo automatically find all possible grains classes, it also triggered lots of warnings from trying to load assemblies that didn't have grain classes (dependency libraries). So the logs were rather noisy. It also slowed the silo startup process. Somebody reported having 150 assemblies in the silo folders and that it would take a minute or longer to unnecessarily scan them all.

That's why in 2.0 we switched to explicit registration of grain assemblies via parts.AddApplicationPart().

[Kritner]

Hey @sergeybykov I was thinking it was probably something like that, but that parts.AddApplicationPart() was actually commented out already in the commit where it was working (I had removed the commented out line in the next iteration, since it was commented out anyway).

[sergeybykov]

Sorry, my bad - I missed that it was already commented out. Is the grain assembly deployed to the silo folder in both cases?

[sergeybykov]

IIRC we still scan the folder in case parts.AddApplicationPart() hasn't been called, and that's when it would depend on whether or not the assembly is in the folder.

[Kritner]

Sorry, my bad - I missed that it was already commented out. Is the grain assembly deployed to the silo folder in both cases?

Indeed, it's a strange one! I made sure to delete the bin folder from the silohost project prior to running the silohost via dotnet run - the grains.dll is being placed in the bin in both examples cases, but only works in the first commit hash.

I've done this on two separate computers using the git checkout [hash], and in both cases the first one works, the second one doesn't.

Again, not a big deal as there's a workaround by using application parts, was just curious if there was anything popping out as obvious as to why it wasn't working.

[sergeybykov]

I think I found the answer. When you add .UseDashboard(), under the covers it calls builder.ConfigureApplicationParts.

That turns off the default behavior of loading and scanning all assemblies in the silo folder that is only applied when application code hasn't configured application parts (to match the previous implicit loading of types). So the gap here is that even though your code doesn't configure application parts, the Dashboard does that, and the runtime doesn't differentiate between the two, which leads to this confusion.

Maybe we should at least treat .AddFrameworkPart() differently from .AddApplicationPart().

[Kritner]

Oh interesting, so AddApplicationPart, once invoked disables the automatic scanning then? Or something similar to that?

I wonder... the workaround I did was to add a new interface to the Grains I had implemented:

Grain.Interfaces

public interface IHelloWorldGrain : IGrainWithGuidKey 
{ 
  Task<string> SayHello(string hello);
}

public interface IAnotherGrain : IGrainWithGuidKey 
{ 
  Task<bool> DoStuff();
}

Grains

public interface IGrainMarker { }

public class HelloWorldGrain : Grain, IHelloWorldGrain, IGrainMarker
{
 // ...
}

public class AnotherGrain : Grain, IAnotherGrain, IGrainMarker
{
 // ...
}

Then configurng all my grains with:

.ConfigureApplicationParts(parts =>
  {
    parts.AddApplicationPart(typeof(IGrainMarker).Assembly).WithReferences();
  )

I'm not sure if the adding of IGrainMarker to each grain is necessary or not - given the method is looking at "assembly" - I guess easy enough to test. But if something like this is required, couldn't the abstract Grain itself implement some sort of grain marker class, so that the user wouldn't need to worry about registering their grains? Or is there a use case in that you wouldn't necessarily want every grain defined to be registered in a silo?

[sergeybykov]

Oh interesting, so AddApplicationPart, once invoked disables the automatic scanning then? Or something similar to that?

Yes, automatic scanning is the fallback mechanism we left, primarily for people upgrading from 1.x where they never specified assemblies to be scanned. If 2.0 was the starting point, we would probably just require at least one call to AddApplicationPart().

I'm not sure your workaround will work. WithReferences() adds dependency assemblies referenced be the given one, not those that reference it. So if you have a grain class that implements IGrainMarker but is defined in a separate assembly, it won't be picked up by parts.AddApplicationPart(typeof(IGrainMarker).Assembly).WithReferences();.

The desired usage is to specify all grain assemblies and all grain interface assemblies (usually by simply adding WithReferences() to the former) explicitly. The fallback scanning is only there for legacy reasons.

[Kritner]

Ah gotcha, yeah I guess at least thus far, I had always been defining my grains in the single assembly, the same where I defined IGrainMarker.

I guess that's all I had, thanks for the assist! :)

[d-barker]

I had a similar issue, two projects, one for the interfaces and one for the implementation. If the auto load option was used (no call to AddApplicationPart(...) ) then it worked, as soon as I added the explicit loading (which is highly recommended) of the assemblies it would fail with a message:

Orleans.Runtime.OrleansConfigurationException : None of the assemblies added to ApplicationPartManager contain generated code for grain interfaces. Ensure that code generation has been executed for grain interface and grain class assemblies and that they have been added as application parts.

I find the message a little miss leading, as all the Orleans build artefacts were being produced and compiled without issue. The issue is the referenced interface and result interfaces are in different assemblies and not 'loadable' unless specified in the AddApplicationPart(); which is obvious now :-)

The solution for me, is to either add both assemblies using the AddApplicationPart() (before build) or use the method WithReferences() as stated by @sergeybykov above.