Repository navigation
1.6.0
1.6.0 has six fixes: with EF Core, jobs left InProgress under an instance name that does not come back are released, and dotnet ef finds the context of UseSharpTickerDbContext; the dashboard works with ticker types of your own, counts the job statuses in the store and shows when each lock holder last locked a job; and a ticker added before the host starts finds its function. The release of those jobs comes with a new EF Core option, SetInProgressLease(TimeSpan), which is on by default with 5 minutes. No database migration for EF Core, no change of the MongoDB collections, no new Redis key. The option is the one addition to the public API and its default the one new default.
New option
SharpTickerEfCoreOptionBuilder.SetInProgressLease(TimeSpan), on by default with 5 minutes (#24).
options.AddOperationalStore(ef =>
{
ef.UseApplicationDbContext<AppDbContext>(ConfigurationType.UseModelCustomizer);
ef.SetInProgressLease(TimeSpan.FromMinutes(5)); // TimeSpan.Zero turns it off
});- With the lease on, every instance that runs the background services does this every fifth of the lease (every minute by default), the first time one fifth of the lease after the start: it sets
LockedAtto now on the jobs and cron occurrences it holdsInProgresssince it started (UpdatedAtis not changed), and it releases the jobs and cron occurrences that areInProgressunder another instance name and whoseLockedAtis older than the lease. A released job isIdlewith no lock holder and noLockedAt, as after the startup release, and the fallback runs it again; a chain child waiting for it follows by its run condition. - Accepted values:
TimeSpan.Zero, which turns the lease off, or a value from 1 second up to 30 days. Any other value throwsArgumentOutOfRangeException. - Keep the lease well above the time a renewal pass can take on your database. Set the same lease on every instance that runs the background services, and turn it off on all of them or on none.
- The lease is part of the EF Core store only. It also runs with Redis next to the store: the heartbeat release frees only the names registered in Redis, the lease also frees names that did not write a heartbeat there; both write the same release. A host with
DisableBackgroundServices()neither renews nor releases. A custom provider that does not derive from the EF Core provider of SharpTicker has no lease; the log says so once atInformation. - The log category is
SharpTicker.EntityFrameworkCore.InProgressLease:Informationwith the number of released jobs and the names that held them (the ten with the most jobs),Warningwhen a renewal or a release fails; the pass is tried again at the next interval.
See Storage and When an instance stops.
Fixed
EF Core
- A job left
InProgressunder an instance name that does not come back is released (#24). With the EF Core store and no Redis, a job that wasInProgresswhen its instance ended without a signal (killed, out of memory, a lost machine) was released only when an instance started again under the same name. A replaced container or a pod of a Deployment gets a new name, so such a job stayedInProgressfor good, and a chain child waiting for it waited too. A running instance now releases it once its lease has ended; see "New option". dotnet effinds the context ofUseSharpTickerDbContext(#89).UseSharpTickerDbContextregistered anIDbContextFactory<TContext>and a scoped context, but not the non-genericDbContextOptionsthrough which the EF Core tools find a context in the application's services, so in a project with no migrations for itdotnet ef migrations addstopped with "No DbContext was found in assembly ...". That registration is now added, before the other registrations, so an application context that takes the non-genericDbContextOptionsstill gets its own. Name the project that holds the migrations withMigrationsAssembly, as Storage shows.
Dashboard
- Lists, add and update work with ticker types of your own (#95). With
AddSharpTicker<TTimeTicker, TCronTicker>, the time ticker lists, the occurrence lists of a cron and adding and updating a ticker failed withNotSupportedExceptionon every store, because the dashboard's JSON options knew only the library's types. When reflection-based serialization is on, a type the dashboard's own context does not know is now read and written through reflection; the library's types still go through the context. An edit in the dashboard keeps the stored value of each public property of your own type that the body does not send; the dashboard's dialogs send only the library's fields, and on EF Core an update writes every column, so without this an edit would empty them. A body that sends such a property still sets it. - The status counts are counted by the store (#96). The time ticker chart (
/time-tickers/graph-data), the cron chart (/cron-tickers/graph-data) and the overall statistics (/ticker/statuses/get) read every visible row, on EF Core with its children or its cron, and grouped them in memory. They now count each status throughTimeTickersQuery()andCronTickerOccurrencesQuery(): on EF Core and MongoDB the database counts. The in-memory and Redis stores read the rows once and count them in memory, as before. The jobs per machine (/ticker/machine/jobs) still read the locked rows, without their children or cron. The counts are made of the same rows as before: top-level time tickers, the visibility rule applied, the occurrences of the visible crons. - The Lock holders panel shows when each name last locked a job and, with Redis, whether it has a heartbeat (#31). The Machines panel listed every node identifier that had locked a job still in the store, ordered by the number of jobs, so the names of stopped or replaced containers with many old rows came first and looked like running machines. The panel is now called Lock holders. Each name shows its number of jobs and how long ago it last locked one, latest first; with Redis also
heartbeatorno heartbeat. The EF Core, MongoDB and in-memory stores keep no record of which nodes run, so there the panel shows the names and their latest lock only. No name is hidden.
Startup
- A ticker added after
UseSharpTickerand before the host starts finds its function (#75). The function registry was built only when the host started, so a time or cron ticker added betweenapp.Build()andapp.Run(), or by a hosted service that starts before SharpTicker's, failed with "Cannot find TickerFunction with name ...".UseSharpTickernow builds the registry; the build has no I/O. The start builds it again, so functions mapped afterUseSharpTickerare found from the start on, and a cron from the configuration that is not a valid expression still stops the start.
Behaviour changes to check before upgrading
- EF Core: old
InProgressjobs run again (#24). At the first pass after the upgrade, about a minute after the start with the default lease, every job and cron occurrence that isInProgressunder another instance name with aLockedAtolder than the lease is released and runs again, however old it is: jobs left by replaced containers or pods can be weeks old, and cron occurrences run late. A chain child waiting for such a job follows by its run condition. Look for these jobs before the upgrade: the startup log of 1.5.0 and later names the instance names that hold them (Information, categorySharpTicker.EntityFrameworkCore.StartupRelease); in the database they are the rows withStatus2 (InProgress) inTimeTickersandCronTickerOccurrences, with theirLockHolderandLockedAt. Delete the ones that should not run (Force delete on the dashboard for a time job), or upgrade withSetInProgressLease(TimeSpan.Zero)and turn the lease on once they are dealt with. A row with noLockedAtor no lock holder is not released by the lease. - EF Core: the same lease on every instance (#24). An instance of an earlier version, or one with the lease off, does not renew its locks, so an instance with the lease on releases the jobs that instance runs for longer than the lease, and they run twice. With different values, the shorter lease releases the jobs of the others earlier than their lease expects. Upgrade all instances together.
- EF Core: "Locked At" shows the last renewal (#24). For a job that is
InProgress,LockedAtand the dashboard's "Locked At" show when the lock was last renewed, not when the job was claimed; so does the latest lock of a name in the Lock holders panel. - EF Core:
dotnet efneeds--contextin a project with a context of its own (#89). In a project that registers a context of its own (AddDbContext<AppDbContext>) and alsoUseSharpTickerDbContext, with no migrations forSharpTickerDbContext, the tools picked the application's context up to 1.5.2 when no--contextwas given; now they find both and stop with "More than one DbContext was found". Scripts that rundotnet ef database update,dotnet ef migrations scriptordotnet ef migrations bundlein such a project need--context AppDbContext. The options action given toUseSharpTickerDbContextalso runs once more, when the new registration is resolved. - Dashboard: the public properties of your own ticker types are shown and accepted (#95). The public properties that your
TTimeTickerandTCronTickeradd are in the dashboard's answers and are read from the bodies of add and update. Mark a property[JsonIgnore]to keep it out of both; such a property keeps its stored value on an edit. - Dashboard: the request check of typed requests (#95). The request-match value of
/ticker-request/{id}(MatchType, the badge in the time ticker list) is 1 where it was 2 (red) for every request that reads as its type, in every application with typed requests: up to 1.5.2 the check failed on the request type itself. - Dashboard: reflection off (#95). In a trimmed or AOT application, where the serializer has reflection switched off, the requests with your own ticker types fail as they did up to 1.5.2.
- Dashboard: a custom provider's query objects are now used for the counts (#96). The count endpoints call
TimeTickersQuery()andCronTickerOccurrencesQuery()withWhereandCountAsync. A customITickerPersistenceProviderwhoseTimeTickersQuery()throws makes/time-tickers/graph-data,/ticker/statuses/getand/ticker/machine/jobsfail; one whoseCronTickerOccurrencesQuery()throws makes/cron-tickers/graph-data,/ticker/statuses/getand/ticker/machine/jobsfail. - Dashboard: the counts are not one snapshot (#96). Each status is counted by a query of its own, so jobs that change status while the counts are made can be counted under two statuses or under none.
- Dashboard: the overall statuses are in enum order (#96).
/ticker/statuses/getlists the statuses that occur in the order ofTickerStatus, where the order was the order in which they were found. The dashboard's UI looks them up by name. - Dashboard: Lock holders and
/ticker/machine/jobs(#31). The panel called Machines up to 1.5.2 is called Lock holders. The answer of/ticker/machine/jobsis ordered by the latest lock of each name, latest first, where it was ordered by the number of jobs. It keepsitem1(the name) anditem2(the number of jobs) and addslastLockedAt(the latest lock, in UTC) andhasHeartbeat(with Redis whether the node has a heartbeat entry; null without Redis, forunknownand when Redis could not be read). A script that relies on the old order can sort byitem2itself. An instance with the background services off still writes its heartbeat with Redis, soheartbeatsays that the instance runs, not that it runs jobs. - Startup:
UseSharpTickerbuilds the function registry (#75). A%Key%cron placeholder is read from the configuration whenUseSharpTickerruns, not when the host starts; a placeholder whose value is not a cron expression is read again at the start, and the start stops on it, as before. A function mapped afterUseSharpTickeris found from the start on.
Upgrade
From 1.5.2: no schema or data step. With EF Core, look for the InProgress jobs under names no instance runs before the upgrade, and delete the ones that should not run, or upgrade with the lease off; upgrade all instances together, with the same lease. Add --context to the dotnet ef commands of a project that also has a context of its own.
Known limits
- EF Core lease. An instance that is still running but cannot renew for the whole lease, because it alone is cut off from the database or its process was paused that long, has its jobs released by the others while they may still run on it; the second run's outcome is stored, the late outcome of the first is not. After an outage of the database longer than the lease, the next pass may release jobs that are still running before their instance renews them.
LockedAtis written with the clock of the renewing instance and compared with the clock of the releasing one, so a difference between the clocks shortens or lengthens the lease by that much. MongoDB has no release step, as before. - Dashboard with your own ticker types. Reflection off is not supported (see above). The in-memory store does not keep the properties your types add. The stored values are kept for the edited ticker only, not for children sent in the body of a time ticker update.
- Lock holders. Without Redis the panel cannot tell a name that runs from one that is gone. The jobs per machine still read the locked rows of both tables.
- Otherwise unchanged from 1.5.2; see its notes.