Skip to content

1.6.0

Choose a tag to compare

@msx752 msx752 released this 15 Nov 09:58
d960ef5

1.6.0 has six fixes: with EF Core, jobs left InProgress under an instance name that does not come back are released, and dotnet ef finds the context of UseSharpTickerDbContext; the dashboard works with ticker types of your own, counts the job statuses in the store and shows when each lock holder last locked a job; and a ticker added before the host starts finds its function. The release of those jobs comes with a new EF Core option, SetInProgressLease(TimeSpan), which is on by default with 5 minutes. No database migration for EF Core, no change of the MongoDB collections, no new Redis key. The option is the one addition to the public API and its default the one new default.

New option

SharpTickerEfCoreOptionBuilder.SetInProgressLease(TimeSpan), on by default with 5 minutes (#24).

options.AddOperationalStore(ef =>
{
    ef.UseApplicationDbContext<AppDbContext>(ConfigurationType.UseModelCustomizer);
    ef.SetInProgressLease(TimeSpan.FromMinutes(5)); // TimeSpan.Zero turns it off
});
  • With the lease on, every instance that runs the background services does this every fifth of the lease (every minute by default), the first time one fifth of the lease after the start: it sets LockedAt to now on the jobs and cron occurrences it holds InProgress since it started (UpdatedAt is not changed), and it releases the jobs and cron occurrences that are InProgress under another instance name and whose LockedAt is older than the lease. A released job is Idle with no lock holder and no LockedAt, as after the startup release, and the fallback runs it again; a chain child waiting for it follows by its run condition.
  • Accepted values: TimeSpan.Zero, which turns the lease off, or a value from 1 second up to 30 days. Any other value throws ArgumentOutOfRangeException.
  • Keep the lease well above the time a renewal pass can take on your database. Set the same lease on every instance that runs the background services, and turn it off on all of them or on none.
  • The lease is part of the EF Core store only. It also runs with Redis next to the store: the heartbeat release frees only the names registered in Redis, the lease also frees names that did not write a heartbeat there; both write the same release. A host with DisableBackgroundServices() neither renews nor releases. A custom provider that does not derive from the EF Core provider of SharpTicker has no lease; the log says so once at Information.
  • The log category is SharpTicker.EntityFrameworkCore.InProgressLease: Information with the number of released jobs and the names that held them (the ten with the most jobs), Warning when a renewal or a release fails; the pass is tried again at the next interval.

See Storage and When an instance stops.

Fixed

EF Core

  • A job left InProgress under an instance name that does not come back is released (#24). With the EF Core store and no Redis, a job that was InProgress when its instance ended without a signal (killed, out of memory, a lost machine) was released only when an instance started again under the same name. A replaced container or a pod of a Deployment gets a new name, so such a job stayed InProgress for good, and a chain child waiting for it waited too. A running instance now releases it once its lease has ended; see "New option".
  • dotnet ef finds the context of UseSharpTickerDbContext (#89). UseSharpTickerDbContext registered an IDbContextFactory<TContext> and a scoped context, but not the non-generic DbContextOptions through which the EF Core tools find a context in the application's services, so in a project with no migrations for it dotnet ef migrations add stopped with "No DbContext was found in assembly ...". That registration is now added, before the other registrations, so an application context that takes the non-generic DbContextOptions still gets its own. Name the project that holds the migrations with MigrationsAssembly, as Storage shows.

Dashboard

  • Lists, add and update work with ticker types of your own (#95). With AddSharpTicker<TTimeTicker, TCronTicker>, the time ticker lists, the occurrence lists of a cron and adding and updating a ticker failed with NotSupportedException on every store, because the dashboard's JSON options knew only the library's types. When reflection-based serialization is on, a type the dashboard's own context does not know is now read and written through reflection; the library's types still go through the context. An edit in the dashboard keeps the stored value of each public property of your own type that the body does not send; the dashboard's dialogs send only the library's fields, and on EF Core an update writes every column, so without this an edit would empty them. A body that sends such a property still sets it.
  • The status counts are counted by the store (#96). The time ticker chart (/time-tickers/graph-data), the cron chart (/cron-tickers/graph-data) and the overall statistics (/ticker/statuses/get) read every visible row, on EF Core with its children or its cron, and grouped them in memory. They now count each status through TimeTickersQuery() and CronTickerOccurrencesQuery(): on EF Core and MongoDB the database counts. The in-memory and Redis stores read the rows once and count them in memory, as before. The jobs per machine (/ticker/machine/jobs) still read the locked rows, without their children or cron. The counts are made of the same rows as before: top-level time tickers, the visibility rule applied, the occurrences of the visible crons.
  • The Lock holders panel shows when each name last locked a job and, with Redis, whether it has a heartbeat (#31). The Machines panel listed every node identifier that had locked a job still in the store, ordered by the number of jobs, so the names of stopped or replaced containers with many old rows came first and looked like running machines. The panel is now called Lock holders. Each name shows its number of jobs and how long ago it last locked one, latest first; with Redis also heartbeat or no heartbeat. The EF Core, MongoDB and in-memory stores keep no record of which nodes run, so there the panel shows the names and their latest lock only. No name is hidden.

Startup

  • A ticker added after UseSharpTicker and before the host starts finds its function (#75). The function registry was built only when the host started, so a time or cron ticker added between app.Build() and app.Run(), or by a hosted service that starts before SharpTicker's, failed with "Cannot find TickerFunction with name ...". UseSharpTicker now builds the registry; the build has no I/O. The start builds it again, so functions mapped after UseSharpTicker are found from the start on, and a cron from the configuration that is not a valid expression still stops the start.

Behaviour changes to check before upgrading

  • EF Core: old InProgress jobs run again (#24). At the first pass after the upgrade, about a minute after the start with the default lease, every job and cron occurrence that is InProgress under another instance name with a LockedAt older than the lease is released and runs again, however old it is: jobs left by replaced containers or pods can be weeks old, and cron occurrences run late. A chain child waiting for such a job follows by its run condition. Look for these jobs before the upgrade: the startup log of 1.5.0 and later names the instance names that hold them (Information, category SharpTicker.EntityFrameworkCore.StartupRelease); in the database they are the rows with Status 2 (InProgress) in TimeTickers and CronTickerOccurrences, with their LockHolder and LockedAt. Delete the ones that should not run (Force delete on the dashboard for a time job), or upgrade with SetInProgressLease(TimeSpan.Zero) and turn the lease on once they are dealt with. A row with no LockedAt or no lock holder is not released by the lease.
  • EF Core: the same lease on every instance (#24). An instance of an earlier version, or one with the lease off, does not renew its locks, so an instance with the lease on releases the jobs that instance runs for longer than the lease, and they run twice. With different values, the shorter lease releases the jobs of the others earlier than their lease expects. Upgrade all instances together.
  • EF Core: "Locked At" shows the last renewal (#24). For a job that is InProgress, LockedAt and the dashboard's "Locked At" show when the lock was last renewed, not when the job was claimed; so does the latest lock of a name in the Lock holders panel.
  • EF Core: dotnet ef needs --context in a project with a context of its own (#89). In a project that registers a context of its own (AddDbContext<AppDbContext>) and also UseSharpTickerDbContext, with no migrations for SharpTickerDbContext, the tools picked the application's context up to 1.5.2 when no --context was given; now they find both and stop with "More than one DbContext was found". Scripts that run dotnet ef database update, dotnet ef migrations script or dotnet ef migrations bundle in such a project need --context AppDbContext. The options action given to UseSharpTickerDbContext also runs once more, when the new registration is resolved.
  • Dashboard: the public properties of your own ticker types are shown and accepted (#95). The public properties that your TTimeTicker and TCronTicker add are in the dashboard's answers and are read from the bodies of add and update. Mark a property [JsonIgnore] to keep it out of both; such a property keeps its stored value on an edit.
  • Dashboard: the request check of typed requests (#95). The request-match value of /ticker-request/{id} (MatchType, the badge in the time ticker list) is 1 where it was 2 (red) for every request that reads as its type, in every application with typed requests: up to 1.5.2 the check failed on the request type itself.
  • Dashboard: reflection off (#95). In a trimmed or AOT application, where the serializer has reflection switched off, the requests with your own ticker types fail as they did up to 1.5.2.
  • Dashboard: a custom provider's query objects are now used for the counts (#96). The count endpoints call TimeTickersQuery() and CronTickerOccurrencesQuery() with Where and CountAsync. A custom ITickerPersistenceProvider whose TimeTickersQuery() throws makes /time-tickers/graph-data, /ticker/statuses/get and /ticker/machine/jobs fail; one whose CronTickerOccurrencesQuery() throws makes /cron-tickers/graph-data, /ticker/statuses/get and /ticker/machine/jobs fail.
  • Dashboard: the counts are not one snapshot (#96). Each status is counted by a query of its own, so jobs that change status while the counts are made can be counted under two statuses or under none.
  • Dashboard: the overall statuses are in enum order (#96). /ticker/statuses/get lists the statuses that occur in the order of TickerStatus, where the order was the order in which they were found. The dashboard's UI looks them up by name.
  • Dashboard: Lock holders and /ticker/machine/jobs (#31). The panel called Machines up to 1.5.2 is called Lock holders. The answer of /ticker/machine/jobs is ordered by the latest lock of each name, latest first, where it was ordered by the number of jobs. It keeps item1 (the name) and item2 (the number of jobs) and adds lastLockedAt (the latest lock, in UTC) and hasHeartbeat (with Redis whether the node has a heartbeat entry; null without Redis, for unknown and when Redis could not be read). A script that relies on the old order can sort by item2 itself. An instance with the background services off still writes its heartbeat with Redis, so heartbeat says that the instance runs, not that it runs jobs.
  • Startup: UseSharpTicker builds the function registry (#75). A %Key% cron placeholder is read from the configuration when UseSharpTicker runs, not when the host starts; a placeholder whose value is not a cron expression is read again at the start, and the start stops on it, as before. A function mapped after UseSharpTicker is found from the start on.

Upgrade

From 1.5.2: no schema or data step. With EF Core, look for the InProgress jobs under names no instance runs before the upgrade, and delete the ones that should not run, or upgrade with the lease off; upgrade all instances together, with the same lease. Add --context to the dotnet ef commands of a project that also has a context of its own.

Known limits

  • EF Core lease. An instance that is still running but cannot renew for the whole lease, because it alone is cut off from the database or its process was paused that long, has its jobs released by the others while they may still run on it; the second run's outcome is stored, the late outcome of the first is not. After an outage of the database longer than the lease, the next pass may release jobs that are still running before their instance renews them. LockedAt is written with the clock of the renewing instance and compared with the clock of the releasing one, so a difference between the clocks shortens or lengthens the lease by that much. MongoDB has no release step, as before.
  • Dashboard with your own ticker types. Reflection off is not supported (see above). The in-memory store does not keep the properties your types add. The stored values are kept for the edited ticker only, not for children sent in the body of a time ticker update.
  • Lock holders. Without Redis the panel cannot tell a name that runs from one that is gone. The jobs per machine still read the locked rows of both tables.
  • Otherwise unchanged from 1.5.2; see its notes.