Added persistence helpers to the jobs service #22246
Conversation
ref https://linear.app/ghost/issue/ENG-2027/ The email analytics job has had a one-off/custom implementation of using the jobs table to persist `started_at` and `finished_at` timestamps for the analytics jobs. This would be helpful for other scheduled/recurring jobs, so this adds helper fns to the JobsService to do so. The intent is for an entry in the `jobs` table to simply hold the last started/finished times so we don't need to schedule a random time on Ghost boot. This ensures jobs are run daily, etc, in the event of frequent reboots.
WalkthroughThe changes introduce new asynchronous methods in both the Possibly related PRs
✨ Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
ghost/job-manager/lib/JobManager.js(1 hunks)ghost/job-manager/lib/JobsRepository.js(2 hunks)ghost/job-manager/test/job-manager.test.js(2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (7)
- GitHub Check: Ghost-CLI tests
- GitHub Check: Database tests (Node 20.11.1, sqlite3)
- GitHub Check: Database tests (Node 22.13.1, mysql8)
- GitHub Check: Database tests (Node 20.11.1, mysql8)
- GitHub Check: Regression tests (Node 20.11.1, sqlite3)
- GitHub Check: Database tests (Node 18.12.1, mysql8)
- GitHub Check: Regression tests (Node 20.11.1, mysql8)
🔇 Additional comments (9)
ghost/job-manager/test/job-manager.test.js (3)
740-851: Solid test coverage for persistent job data.Overall, the tests in this range comprehensively verify the behavior of:
getLastJobRunTimestamp()setJobTimestamp()setJobStatus()They cover both success and error scenarios, ensuring a well-rounded validation of job timestamps and statuses.
852-951: Good repository test organization.The test cases for
getQueuedJobs,delete, andaddQueuedJobthoroughly check default/custom limits, error handling, and transaction usage. This ensures reliability and correctness when managing queued jobs.
952-1016: Robust error handling tests.These tests effectively confirm:
- Worker messages without a custom handler
- Domain event dispatch
- Promise rejections for both single job and “all jobs” in-flight
They help guarantee the JobManager can gracefully handle errors and signal failures correctly.
ghost/job-manager/lib/JobsRepository.js (2)
3-3: Import usage is appropriate.The import of
IncorrectUsageErroris well-placed for validating timestamp fields.
136-145: Neat retrieval of last run timestamp.The
getLastRunTimestampmethod correctly returnsfinished_atif available, otherwise falling back tostarted_at. Good clarity for “latest” run detection.ghost/job-manager/lib/JobManager.js (4)
477-481: Helper to retrieve job data.
getJobDatais straightforward, returning early if no repository is present.
489-499: Timestamp retrieval integration.
getLastJobRunTimestampneatly delegates to the repository. Looks good for confirming the last known completion or start time.
501-512: Timestamp assignment logic is clear.
setJobTimestampdoes a no-op if the repository is absent to avoid runtime errors in ephemeral environments. This aligns with the intended fallback behavior.
514-524: Clean status update delegation.
setJobStatussimilarly delegates, enabling flexible job status tracking. With the repository fix for ID usage, this logic works nicely.
| /** | ||
| * Updates or creates a job's timestamp | ||
| * @param {string} name - The name of the job | ||
| * @param {'started_at'|'finished_at'} field - The timestamp field to update | ||
| * @param {Date} date - The timestamp value | ||
| */ | ||
| async setTimestamp(name, field, date) { | ||
| if (field !== 'started_at' && field !== 'finished_at') { | ||
| throw new IncorrectUsageError({ | ||
| message: `Invalid timestamp field "${field}". Must be either "started_at" or "finished_at"` | ||
| }); | ||
| } | ||
|
|
||
| const job = await this.read(name); | ||
| if (!job) { | ||
| await this.add({name, [field]: date}); | ||
| } else { | ||
| await this.update(name, {[field]: date}); | ||
| } | ||
| } |
There was a problem hiding this comment.
Likely ID mismatch in update calls.
In setTimestamp, the update function expects the job's ID, but the code calls this.update(name, {…}). It should instead call this.update(job.id, {…}) to ensure the existing job is updated correctly. Otherwise, the edit will target an invalid ID.
- await this.update(name, {[field]: date});
+ await this.update(job.id, {[field]: date});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /** | |
| * Updates or creates a job's timestamp | |
| * @param {string} name - The name of the job | |
| * @param {'started_at'|'finished_at'} field - The timestamp field to update | |
| * @param {Date} date - The timestamp value | |
| */ | |
| async setTimestamp(name, field, date) { | |
| if (field !== 'started_at' && field !== 'finished_at') { | |
| throw new IncorrectUsageError({ | |
| message: `Invalid timestamp field "${field}". Must be either "started_at" or "finished_at"` | |
| }); | |
| } | |
| const job = await this.read(name); | |
| if (!job) { | |
| await this.add({name, [field]: date}); | |
| } else { | |
| await this.update(name, {[field]: date}); | |
| } | |
| } | |
| /** | |
| * Updates or creates a job's timestamp | |
| * @param {string} name - The name of the job | |
| * @param {'started_at'|'finished_at'} field - The timestamp field to update | |
| * @param {Date} date - The timestamp value | |
| */ | |
| async setTimestamp(name, field, date) { | |
| if (field !== 'started_at' && field !== 'finished_at') { | |
| throw new IncorrectUsageError({ | |
| message: `Invalid timestamp field "${field}". Must be either "started_at" or "finished_at"` | |
| }); | |
| } | |
| const job = await this.read(name); | |
| if (!job) { | |
| await this.add({ name, [field]: date }); | |
| } else { | |
| await this.update(job.id, { [field]: date }); | |
| } | |
| } |
| /** | ||
| * Updates or creates a job's status | ||
| * @param {string} name - The name of the job | ||
| * @param {string} status - The status to set | ||
| */ | ||
| async setStatus(name, status) { | ||
| const job = await this.read(name); | ||
| if (!job) { | ||
| await this.add({name, status}); | ||
| } else { | ||
| await this.update(name, {status}); | ||
| } | ||
| } |
There was a problem hiding this comment.
Same ID mismatch in setStatus method.
Similar to setTimestamp, passing name to this.update is incorrect. It should pass the found job's ID, or updates will fail.
- await this.update(name, {status});
+ await this.update(job.id, {status});📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /** | |
| * Updates or creates a job's status | |
| * @param {string} name - The name of the job | |
| * @param {string} status - The status to set | |
| */ | |
| async setStatus(name, status) { | |
| const job = await this.read(name); | |
| if (!job) { | |
| await this.add({name, status}); | |
| } else { | |
| await this.update(name, {status}); | |
| } | |
| } | |
| /** | |
| * Updates or creates a job's status | |
| * @param {string} name - The name of the job | |
| * @param {string} status - The status to set | |
| */ | |
| async setStatus(name, status) { | |
| const job = await this.read(name); | |
| if (!job) { | |
| await this.add({name, status}); | |
| } else { | |
| await this.update(job.id, {status}); | |
| } | |
| } |
ref https://linear.app/ghost/issue/ENG-2027/
The email analytics job has had a one-off/custom implementation of using the jobs table to persist
started_atandfinished_attimestamps for the analytics jobs. This would be helpful for other scheduled/recurring jobs, so this adds helper fns to the JobsService to do so.The intent is for an entry in the
jobstable to simply hold the last started/finished times so we don't need to schedule a random time on Ghost boot. This ensures jobs are run daily, etc, in the event of frequent reboots.This also expands the unit test coverage to ~95% for the job manager lib.