Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate with webjobs SDK #712

Open
Metaphis opened this issue May 16, 2019 · 7 comments

Comments

Projects
None yet
2 participants
@Metaphis
Copy link

commented May 16, 2019

I'm attempting to integrate simpleinjector with the azure webjob sdk

I'm following the guide for the generic webjob integration combined with the webjob sdk guide and I came up with this:

var container = new Container();

var builder = new HostBuilder();

builder.ConfigureWebJobs(b =>
{
    b.AddTimers();
})
.ConfigureServices((hostContext, services) =>
{
    services.AddSimpleInjector(container, options => 
    { 
        options.Services.AddSingleton<IJobActivator>(
            new SimpleInjectorJobActivator(options.Container));
    });
});

var host = builder.Build().UseSimpleInjector(container, options => {});
    host.Run();
    public class SimpleInjectorJobActivator : IJobActivator
    {
        private Container _container;

        public SimpleInjectorJobActivator(Container container)
        {
            _container = container;
        }

        public T CreateInstance<T>() => (T)_container.GetInstance(typeof(T));
    }

I then have a triggered job:

public class TimedJobTrigger
{
    private readonly IJobExecutor _jobExecutor;

    public TimedJobTrigger(
        IJobExecutor jobExecutor)
    {
        _jobExecutor = jobExecutor;
    }

    public Task RunAsync(
        [TimerTrigger("00:01:00", RunOnStartup = false)] TimerInfo timerInfo,
        CancellationToken cancellationToken)
    {
        return _jobExecutor.ExecuteAsync(cancellationToken);
    }
}

This works fine as long as none of the registered services are scoped either from the framework container or from the simple injector container. If I have any scoped dependencies then I'm stuck.

I dug through the webjob code and the way the scopes are created is extremely convoluted.

The FunctionExecutor gets passed in a IFunctionInstanceEx. In general I think it is a FunctionInstanceWrapper.

Then this call to FunctionInstanceWrapper.InstanceServices creates a scope which gets disposed when the wrapper is disposed.

I'm looking for guidance on the best way to go about this.
A) I could hook into the IServiceProvider somehow, and implement CreateScope() using the async scope hoping that the flow works properly.
B) I could try replacing some of the components but I'm not even sure where I'd even get started.

Any idea?

@Metaphis Metaphis added the question label May 16, 2019

@dotnetjunkie dotnetjunkie changed the title Integrate SimpleInjector with webjobs SDK Integrate with webjobs SDK May 16, 2019

@dotnetjunkie

This comment has been minimized.

Copy link
Collaborator

commented May 16, 2019

I don't think it is possible to integrate any DI Container with WebJobs at the moment, as you can read in more detail in my (just posted) comment on the azure-webjobs-sdk repository.

The only feasible solution I can see at this moment is to use your Job implementations as Humble Objects. This basically means that you:

  • extract all interesting logic out of a job,
  • move it into a different service
  • resolve that service from within the job's method(s), while wrapping it in an active scope.

This might look as follows:

public class TimedJobTrigger
{
    private readonly Container _container;

    public TimedJobTrigger(Container container) => _container = container;

    public async Task RunAsync(
        [TimerTrigger("00:01:00", RunOnStartup = false)] TimerInfo timerInfo,
        CancellationToken cancellationToken)
    {
        using (AsyncScopedLifestyle.BeginScope(_container)
        {
            var service = _container.GetInstance<IJobExecutor>();
            return await service.ExecuteAsync(cancellationToken);
        }
    }
}

This, obviously will lead to a lot of code duplication, but if you define a common abstraction around your services, you will be able to simplify this model. For instance:

public interface IJob<TData>
{
    Task RunAsync(TData data, CancellationToken cancellationToken);
}

In this case you change the IJobExecutor implementation to the following:

public class MyJobExecutor : IJob<EmptyJobData>
{
    Task RunAsync(EmptyJobData data, CancellationToken cancellationToken)
    {
       await ...
    }
}

public class EmptyJobData
{
    public static readonly EmptyJobData Instance = new EmptyJobData();
}

On top of this IJob<T> abstraction, you can now define a helper method that ensures creation of a scope, resolving the job and executing the job:

public static class JobContainerExtensions
{
    public static Task Handle<TJob>(
        this Container container, CancellationToken token)
        where TJob : IJob<EmptyJobData> =>
        container.Handle<TJob, EmptyJobData>(EmptyJobData.Instance, token);

    public static Task Handle<TJob, TData>(
        this Container container, TData data, CancellationToken token)
        where TJob : IJob<TData>
    {
        using (AsyncScopedLifestyle.BeginScope(container))
        {
            TJob job = container.GetInstance<TJob>();
            return await job.ExecuteAsync(data, token);
        }
    }    
}

This allows you to minimize your Azure WebJobs functions to the following:

public class TimedJobTrigger
{
    private readonly Container _container;

    public TimedJobTrigger(Container container) => _container = container;

    public async Task RunAsync(
        [TimerTrigger("00:01:00", RunOnStartup = false)] TimerInfo timerInfo,
        CancellationToken token) =>
        _container.Handle<MyJobExecutor>(token);
}

As the TimedJobTrigger is now a Humble Object, which can be considered part of the Composition Root, it is fine to move away from constructor injection, and instead store the Container in a static field:

public static JobHandler
{
    private static Container container; // create and configure container
    
    public static Task Handle<TJob>(CancellationToken token) => ...
}

// Simplifies your Humble Objects
public class TimedJobTrigger
{
    public async Task RunAsync(
        [TimerTrigger("00:01:00", RunOnStartup = false)] TimerInfo timerInfo,
        CancellationToken token) =>
        JobHandler.Handle<MyJobExecutor>(token);
}

You might even be able to simplify your jobs even more, as they are now always supplied with a CancellationToken, but most of the implementations will typically not need such cancellation token at all. You can, therefore, move this CancellationToken out of the public method of the IJob<T> API, and instead inject a value into the constructor for those implementations that truly require it:

// Application code
public interface IJob<TData>
{
    Task RunAsync(TData data); // no more CancellationToken
}

// Hide CancellationToken behind a 'provider' abstraction
public interface ICancellationTokenProvider
{
    CancellationToken Token { get; }
}

// Inject ICancellationTokenProvider into the constructor
public class MyJobExecutor : IJob<EmptyJobData>
{
    private readonly ICancellationTokenProvider _tokenProvider;
    public MyJobExecutor(ICancellationTokenProvider tokenProvider) =>
        _tokenProvider = tokenProvider;

    Task RunAsync(EmptyJobData data)
    {
        // provide it to other services if needed
        _tokenProvider.Token.Cancel();
        await ...
    }
}

Your Composition Root will now look as follows:

public sealed class ScopedCancellationTokenProvider : ICancellationTokenProvider
{
    public CancellationToken Token { get; set; }
}

// Registrations
container.Register<ScopedCancellationTokenProvider>(Lifestyle.Scoped);
container.Register<ICancellationTokenProvider, ScopedCancellationTokenProvider>(
    Lifestyle.Scoped);
    
// Full handle method
public static Task Handle<TJob, TData>(TData data, CancellationToken token)
    where TJob : IJob<TData>
{
    using (AsyncScopedLifestyle.BeginScope(container))
    {
        var provider = container.GetInstance<ScopedCancellationTokenProvider>();
        
        // Set the token.
        provider.Token = token;
    
        TJob job = container.GetInstance<TJob>();
        
        // No more token.
        return await job.ExecuteAsync(data);
    }
}

These are just some ideas you can play with.

I hope this helps.

@Metaphis

This comment has been minimized.

Copy link
Author

commented May 16, 2019

Thanks a lot, this is super helpful. I'll also monitor the webjobs SDK thread but I doubt we'll get traction anytime soon.

@dotnetjunkie

This comment has been minimized.

Copy link
Collaborator

commented May 16, 2019

Hi @Metaphis,

We got a response from the Azure team, and it seems the right hooks are actually in place, I just happen to mis them.

Would you be able to try the following registration?

services
    .AddScoped<SimpleInjectorJobActivator.ScopeDisposable>()
    .AddSingleton<IJobActivator>(new SimpleInjectorJobActivator(container));

With the following SimpleInjectorJobActivator implementation:

public class SimpleInjectorJobActivator : IJobActivatorEx
{
    private readonly Container container;

    public SimpleInjectorJobActivator(Container container) => this.container = container;

    public T CreateInstance<T>(IFunctionInstanceEx functionInstance)
    {
        var disposer = functionInstance.InstanceServices.GetRequiredService<ScopeDisposable>();
        this.disposer.Scope = AsyncScopedLifestyle.BeginScope(this.container);
        return (T)this.container.GetInstance(typeof(T));
    }

    // Ensures a created Simple Injector scope is disposed at the end of the request
    public sealed class ScopeDisposable : IDisposable
    {
        public Scope Scope { get; set; }
        public void Dispose() => this.Scope?.Dispose();
    }
}

Please let me know if that works out for you.

@Metaphis

This comment has been minimized.

Copy link
Author

commented May 16, 2019

I wrote a small sample based on the suggested implementation and verified that scopes get resolved correctly.
The following nugets are needed:
Microsoft.Azure.WebJobs
Microsoft.Azure.WebJobs.Extensions
SimpleInjector.Integration.GenericHost

Put this in the main method

static void Main(string[] args)
{
    var container = new Container();

    var builder = new HostBuilder();

    builder.ConfigureWebJobs(webJobBuilder =>
    {
        webJobBuilder
            .AddAzureStorageCoreServices()
            .AddTimers();
    })
    .ConfigureServices((hostContext, services) =>
    {
        services
            .AddScoped<SimpleInjectorJobActivator.ScopeDisposable>()
            .AddSingleton<IJobActivator>(new SimpleInjectorJobActivator(container));

        services.AddSimpleInjector(container, options =>
        {
            options.Container.Register<IService1, DisposableService1>(Lifestyle.Scoped);
            options.Services.AddScoped<IService2, DisposableService2>();
        });
    });

    var host = builder.Build().UseSimpleInjector(container, options => { });

    using (host)
    {
        host.Run();
    }
}

The Job activator looks like this:

public class SimpleInjectorJobActivator : IJobActivatorEx
{
    private readonly Container container;

    public SimpleInjectorJobActivator(Container container) => this.container = container;

    public T CreateInstance<T>(IFunctionInstanceEx functionInstance)
    {
        var disposer = functionInstance.InstanceServices.GetRequiredService<ScopeDisposable>();
        disposer.Scope = AsyncScopedLifestyle.BeginScope(container);
        return (T)container.GetInstance(typeof(T));
    }

    public T CreateInstance<T>()
    {
        //This will never get called because we're implementing IJobActivatorEx
        throw new NotSupportedException("Cannot create an instance outside of scopes");
    }

    // Ensures a created Simple Injector scope is disposed at the end of the request
    public sealed class ScopeDisposable : IDisposable
    {
        public Scope Scope { get; set; }
        public void Dispose() => Scope?.Dispose();
    }
}

A trigger can then be defined this way:

public class TimedTrigger 
{
    private readonly IService1 s1;
    private readonly IService2 s2;
    public TimedTrigger(IService1 s1, IService2 s2)
    {
        this.s1 = s1;
        this.s2 = s2;
    }

    public void DoTimedStuff([TimerTrigger("00:00:30")]TimerInfo timer)
    {
    }
}

A couple of notes
A) it's is not currently possible to set container.Options.ResolveUnregisteredConcreteTypes = false; sincr SimpleInjector will not be able to resolve the concrete class.

It's also not possible to register those classes automatically since there are multiple types of triggers and they each have their own implementation when selecting the classes.

In addition the ResolveUnregisteredType event doesn't seem to include information that would allow us to limit concrete type resolution to the root resolution only.

B) The trigger can implement IDisposable and the WebJobs framework will dispose of the object. I would advice against doing that and leave your trigger class transient.

@dotnetjunkie

This comment has been minimized.

Copy link
Collaborator

commented May 17, 2019

A) it's is not currently possible to set container.Options.ResolveUnregisteredConcreteTypes = false; since SimpleInjector will not be able to resolve the concrete class.

Not completely true, but you will manually ensure that all jobs are registered. Registration of all root types is advised anyway. You will have to find a convention for that. For instance, end all job class names with "Job" or let them implement an IJob interface. With the IJob convention you can do the following:

var jobAssembly = typeof(TimedTrigger).Assembly;
var types = container.GetTypesToRegister<IJob>(jobAssembly);
types.ToList().ForEach(concreteType => container.Register(concreteType));

With the "Job" convention, it is a bit more work, but would like as follows:

var jobAssembly = typeof(TimedTrigger).Assembly;

var types =
    from type in jobAssembly.GetTypes()
    where type.Name.EndsWith("Job")
    where !type.IsAbstract && !type.IsGenericTypeDefinition
    select type;

types.ToList().ForEach(concreteType => container.Register(concreteType));

As another convention, you might also reflect over the type's public methods to see whether they are marked with WebJob SDK-specific attributes. A lot options to choose from :)

In addition the ResolveUnregisteredType event doesn't seem to include information that would allow us to limit concrete type resolution to the root resolution only.

That is correct. Such feature was actually discussed prior to v4.5 (see this) and I decided not to make it possible.

B) The trigger can implement IDisposable and the WebJobs framework will dispose of the object. I would advice against doing that and leave your trigger class transient.

I would say it is a design flaw when WebJobs disposes the trigger that it did not create that class itself. When it delegates the creation of a trigger to an IJobActivator, it should not do the disposing itself, but let this be handled by the IJobActivator.

As you know, in case you register a Transient disposable object in Simple Injector, you get a warning when you call Verify(), because Simple Injector does not track (and thus dispose) transient components. You could ignore such warning programmatically when you are sure that WebJobs disposes these types, but I would say that in a properly designed system, you should hardly ever need to have those root objects implement IDisposable. Those root objects typically only delegate work on a more high-level way, while only lower-level components (database connections etc) need disposal.

I'm interested to see some examples of when you need dispose logic in your jobs and triggers.

@Metaphis

This comment has been minimized.

Copy link
Author

commented May 17, 2019

Ah I didn't think about adding my own convention on top of triggers. That's a good idea.

As for having disposable trigger classes I meant that I don't think having a disposable trigger is a good idea anyway. In our systems the trigger is just that a trigger that then calls a service when triggered with no extra logic.

@dotnetjunkie

This comment has been minimized.

Copy link
Collaborator

commented May 17, 2019

As for having disposable trigger classes I meant that I don't think having a disposable trigger is a good idea anyway.

Agreed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.