You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am trying to fetch a large amount of data and thought to use bull to help handling it.
I am quite new to bull but you will find below roughly how I dit it.
However, the job is moved to complete when the fetch action is done.
The job will then pipe the stream and process the data.
constworker=newWorker('downloads',asyncjob=>{constresponseStream=awaitfetch(job.data.url);responseStream.body.pipe(ndjson.parse()).on('data',asynccurrentGame=>{constgame=awaitgenerateGame(currentGame);awaitgame.populate('user');awaitgame.save().on('pause',()=>{console.log('pause');}).on('end',()=>{console.log('end');}).on('error',error=>{console.log(newError(error));});},{connection});worker.on('completed',job=>{console.log(`${job.id} has completed!`);});
The text was updated successfully, but these errors were encountered:
Your second statement, starting with responseStream.body, does not return a promise. Hence, the worker finishes immediately. You want to either return a promise or find something you can await.
Do you know where can I find documentation on this? When does the job know it should end? Is there any option to make the job open until I manually move it to completed ?
I will admit I am having a hard time implementing this.
I update the code but the completed event is still triggered before any progress.
Hi,
I am trying to fetch a large amount of data and thought to use bull to help handling it.
I am quite new to bull but you will find below roughly how I dit it.
However, the job is moved to complete when the fetch action is done.
The job will then pipe the stream and process the data.
The text was updated successfully, but these errors were encountered: