Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamically add childJobs while the main job is running #599

Closed
dotMortis opened this issue Jun 17, 2021 · 2 comments
Closed

Dynamically add childJobs while the main job is running #599

dotMortis opened this issue Jun 17, 2021 · 2 comments

Comments

@dotMortis
Copy link

Maybe I missed it, but it would be awesome if it was possible to add childJobs to a job while it is running.

Example:

const mainProcessor = async (job: Job) => {
  try {
    // do something
  } catch (error) {
    await job.addChild('errorProcessing', {error: error, parentId: job.id});
    throw error; 
  }
}
@roggervalf
Copy link
Collaborator

@Haruki-Mortis you can do it by using https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.job.create.md that has JobOptions.parent https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.jobsoptions.md, as I understand by your example, when you create that child, your parent job will not move to failed since you will have a pending child, at some point you would need to move your parent job to waiting children state https://github.com/taskforcesh/bullmq/blob/master/src/classes/job.ts#L801, there is a method to do this, but it is used on manual processing, you could use it but you should be aware that it will throw an error when the worker will try to move the parent job to failed or completed, then the parent job will be re-processed when the job is returned to active state. For now there is not an atomic way to manage this, but it is still in discussion about how to design it.

@dotMortis
Copy link
Author

@roggervalf Thank you very much for the answer.
It helps me a lot.
In my current case, I don't have to wait for the child to be processed.
In the first case it is important to see all child/parent jobs in the job information.

roggervalf added a commit to roggervalf/bullmq that referenced this issue Jul 1, 2021
there is a case where we only want to keep the relationship between parent-child and not waiting for
children, so it is posible that parent completes first

re taskforcesh#599
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants