-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Jobs after creation are not adding to the database #23
Comments
Thank you for the issue! |
Maybe a |
Just an idea. Maybe setting up the |
I'm not using the Discuss feature, but I'm going to open Discord as soon as possible. @kostysh |
The https://docs-pulse.pulsecron.com/docs/creating-jobs/create#example-usage |
Thank you. Currently, jobs are being added to the database. Which is a use case when you do not need to store jobs in a database? I am not familiar with the code yet but I was not able to find a place where saved jobs are restored from the database to the queue at the start. As I am right, to be processed at the start time jobs must be populated in the |
If you are looking for a method that saves immediately why not refer to this? example await pulse.start();
await pulse.every('1 minutes', 'delete old users', { description: 'test' });
// or await pulse.schedule('in 1 minutes', 'delete old users', { to: 'admin@example.com' }); |
In my case, I have to process a series of tasks, each of which can take approximately 30 seconds to 2 minutes. These tasks are not recurrent and should not be scheduled. I just want to be sure, that if this task has been enqueued it will be guaranteed to be processed and the processing result will be logged. |
Also, having all the jobs in the memory is not a good idea from a scalability perspective. |
Such a design approach helps to manage the application’s logic and data flow more clearly, empowering users to take a more active role in managing the state of data, thereby leading to more stable and predictable applications
I don't understand all of your situation, but can the following example be your solution? await pulse.start();
const job = pulse.create('delete old users', { to: 'pulsecron@gmail.com' });
await job.save();
// write your own logic...
job.repeatEvery('10 minutes');
job.unique({ 'data.type': 'email', 'data.userId': '12345' });
await job.save(); |
You can also specify the 'concurrency' option as a small number so that only a certain number of jobs can be into memory. https://docs-pulse.pulsecron.com/docs/defining-job-processors#parameters |
Ok, I will try to be clear with the example: interface MyDataType {
//...
}
pulse.define<MyDataType>('processData', /***/); // job processor
// This helper can be called by API method at any time
function createJob <T extends MyDataType>(pulse: Pulse, data: T) {
const job = pulse.create<T>('processData', data);
await job.save(); // ok, as it is
}
await pulse.start();
// ^--- unprocessed jobs must be restored and processed here I expect that after a server restart, any unprocessed jobs created prior to the restart will be restored from a database and processed. This is a usual practice for queues with persisted jobs. |
In the current case, if the server goes down in the middle and then restarts, nextRunAt will be modified to the next schedule in line with the job schedule rule. But Do you want the job to run again as soon as the server restarts? |
If this job is not been processed - yes. |
I'll implement this as soon as possible and get it. |
Thank you! I guess, we can close this issue. |
I brought you what you wanted! @kostysh I also added discussions in response to your comments. and I will add discord later. |
Nice! I see, |
Hi,
please clarify the
job
workflow.I am adding jobs using
pulse.create('<job_name>', '{job_data}')
.When a job is added without errors, it does not appear in the job collection (I have checked via MongoDB console) and
_id
parameter of ajob
isundefined
.If a server goes down at this moment all the added jobs will be missed after the server restart.
How do you manage jobs to make it possible to finish all unfinished jobs after the server restart?
By the way, here is my configuration with which I am starting a queue:
The text was updated successfully, but these errors were encountered: