Hangfire :: Background Jobs

• An Open source framework which helps to create, process and manage background jobs
• Hangfire
• Actual Repo | Forked Repo
• provides a dashboard to monitor/control the Jobs

Types:

• Fire-and-forget jobs

  • Fire-and-forget jobs are executed only once and almost immediately after creation.

  var jobId = BackgroundJob.Enqueue(() => Console.WriteLine("Fire-and-forget!"));
  

  • steps:
    • Serialize method and parameters
    • Create new background job based on the serialized information
    • Save to Job Storage and Queue
    • Return to Caller
    • Hangfire server picks up Job and processes
    • Job status updated

• Delayed jobs

  • Delayed jobs are executed only once too, but not immediately, after a certain time interval.

  var jobId = BackgroundJob.Schedule( () => Console.WriteLine("Delayed!"), TimeSpan.FromDays(7));
  

  • Ensure that ASP.Net application is configured to always run in IIS
  • Hangfire server checks every 15 seconds to see if there are any scheduled jobs

• Recurring jobs

  • Recurring jobs fire many times on the specified CRON schedule.

  RecurringJob.AddOrUpdate("JOB_ID", () => Console.WriteLine("Recurring!"), Cron.Daily);
  

  • Cron types \ Cron expressions used to schedule the recurring time
  • RemoveIfExists(jobId) -> To remove if there is a recurring Job
  • Trigger(jobId) -> trigger the job at any time without affecting the schedule

• Continuations

  • Continuations are executed when its parent job has been finished.

  BackgroundJob.ContinueWith( jobId, () => Console.WriteLine("Continuation!"));
  

• Batches Pro

  • Batch is a group of background jobs that is created atomically and considered as a single entity.

  var batchId = BatchJob.StartNew(x =>
  {
      x.Enqueue(() => Console.WriteLine("Job 1"));
      x.Enqueue(() => Console.WriteLine("Job 2"));
  },
  "Batch jobs" -> Description of batch jobs
  );
  

  • Nuget package Hangfire.Pro
  • Configuration.useBatches() -> to show background jobs in dashboard

• Batch Continuations Pro

  • Batch continuation is fired when all background jobs in a parent batch finished.

  BatchJob.ContinueWith(parentJobId, x =>
  {
      x.Enqueue(() => Console.WriteLine("Last Job"));
  });
  

  • Child batch job starts after Parent job

• Generic Comments

  • .ContinueWith() -> Is available for all jobs Ex: BackgroundJob.ContinueWith() & BatchJob.ContinueWith()
  • Any paramters can be used (should be serializable - json)
  • Reference parameters (ref & out) are not supported (as the memory location may not be available for the background Job)
  • Use concrete values where possible. Ex: Store id and send the same to Job.
  • Passing Dependencies (for injected objects)
    • Preferably make jobs Static Background Methods
    • For reference objects
      • JobActivator creates an instance using default constructor
      • Constructor for Unit Testing

Hangfire Dashboard & Job Storage

• Written as OWIN Middleware (ASP.Net, Nancy, ServiceStack)
• OWIN Self-host (Console app | Windows services)
• Requirements
  • Microsoft.OWIN.Host.SystemWeb
  • OWIN startup class
    • app.UseHangfireDashboard()

Exception Handling

• Retry is triggered (10 times) after certain interval.
  • AutomaticRetry => can be overridden at global & method level
    • Global: GlobalJobFilters.Filters.Add(new AutomaticRetryAttribute { Attempts = 1 });
    • Method: [AutomaticRetry(attempts = 0)] attribute above method
• Details are shared in Dashboard. Under Failed jobs
• Exceptions can be logged. Requires configuration

Logging

• Automatic Logging: Serilog, NLog, Log4Net, EntLib Logging, Loupe and Elmah
  • Log.Logger = new LoggerConfiguration().WriteTo.RollingFile(@"c:\hangfirelogs.txt").CreateLogger();
• Custom Logger: ILogProvider, ILog (using Hangfire.Logging)
  • LogProvider.SetCurrentLogProvider(new CustomHangfireLogProvider());

Cancellation Tokens

• Used when the job is cancelled because of:
  • Hangfire server shutdown
  • Manual cancellation of job
• IJobCancellationToken
  • ThrowIfCancellationRequested()
  • OperationCanceledException

Job Storage

• SQL Server Configuration
  • Polling Interval (default 15 seconds)
  • PrepareSchemaIfNecessary (default true) - Tells Hangfire to create Hangfire DB if it doesn't exists

  car options = new sqlServerStorageOptions {
    QueuePollInterval = TimeSpan.FromSeconds(60),
    PrepareSchemaIfNecessary = false
  };
  GlobalConfiguration.Configuration.UseSqlServerStorage("ConStringName", options);
  

• Redis Support : Hangfire PRO feature
  • For performance
  • For Linux
  • Unofficial windows version
  • Configuration
    • hangfire.Pro.Redis
    • .UseRedisStorage("localhost:6379", 0, options)

Best practices, Scaling out & Performance

• Background Job Code
  • Reentrant methods
    • Interruptible: The job is done at in-memory first and later saved into the database
    • repeatable for retries
  • Method parameters
    • should be small and simple
    • Use primitives datatypes instead of objects - because of the serialization process
      • Send an ID and later load the data in the job code as required
  • Unit tests, IOC and filters
  • Tracking Job Progress - to inform users about the status
    • Polling v/s Pushing (Signal R)
• Unit Tests
  • IBackgroundJobclient: instead of BackgroundJob
  • Enables mocking
• IOC
  • Hangfire.Autofac

  var builder = new ContainerBuilder();
  builder.RegisterType<>().As<I>().WithParameter("paramname", paramObject);
  var container = builder.Build();
  GlobalConfiguration.Configuration.UseAutofacActivator(container);
  

  • Hangfire.Ninject
  • Hangfire.SimpleInjector
  • Hangfire.Windsor

Job Filters

• Certain events can be intercepted
  • Creation of background jobs
  • Start background processing
  • Status change of background jobs
  • Can be applied to
    • Methods
    • Classes
    • Entire application (Global)
• Creating Filter Attribute:
  • JobFilter should inherit JobFilterAttribute
  • IApplyStateFilter -> For Status change events
  • IClientFilter -> OnCreating / OnCreated Jobs
  • IElectStatefilter
  • IServerFilter
• Using Filter Attribute
  • [Custom_JobFilterAttribute] above Class/Method definition
  • Global: GlobalJobFilters.Filters.Add(new Custom_JobFilterAttribute()); in Startup.cs

Deployment

• Hangfire server in an ASP.NET web app
  • Configure IIS web app as always running
    • Asp.Net auto-start web applications
    • Hangfire.io
    • Azure switch on auto-start
  • Separate out Hangfire Server (from Asp.Net app -> may be to a windows service) - remove application dependency
    • Create a windows Service
    • Install TopShelf - Make it run and behave like a Windows Service (Nuget package)
      • Lets us run Windows service as a console application
    • Install Hangfire.Core & hangfire.SqlServer (Nuget package)
    • Install TestProject.Data , TestProject.Models , TestProject.OptimizationEngine
    • Install Logging Framework (Nuget package)
  • Multiple instances of console application can be started for Scaling-out
    • Start => _server = new BackgroundJobServer | Stop => _server.Dispose();
• Multiple Queues
  • Queues with different priorities
  • Split queues by server
  • How
    • Background job code set Queue Attribute

    [Queue("critical")] -> Class, Method
    

    • BackgroundServer options

    var options = new BackgroundJobServerOptions {
      Queues = new[] { "critical", "default" }
    };
    app.UsehangfireServer(options);
    

• Configuring Worker Threads (# => number of jobs that can run in parallel)
  • By default Hangfire senses the number of worker threads and uses it new BackgroundJobServerOptions { WorkerCount = 30 };