Skip to content

Commit

Permalink
docs(bullmq-pro): add v6 api references (#2106)
Browse files Browse the repository at this point in the history
  • Loading branch information
roggervalf committed Aug 2, 2023
1 parent 0027145 commit 235ff47
Show file tree
Hide file tree
Showing 5 changed files with 50 additions and 34 deletions.
52 changes: 28 additions & 24 deletions docs/gitbook/bullmq-pro/batches.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,18 @@ Workers using batches have slightly different semantics and behavior than normal
In order to enable batches you must pass the batch option with a size representing the maximum amount of jobs per batch:

```typescript
const worker = new WorkerPro(
'My Queue',
async (job: JobPro) => {
const batch = job.getBatch();

const worker = new WorkerPro("My Queue", async (job: JobPro) => {
const batch = job.getBatch();

for(let i=0; i<batch.length; i++) {
for (let i = 0; i < batch.length; i++) {
const batchedJob = batch[i];
await doSomethingWithBatchedJob(batchedJob);
}
}, { connection, batches: { size: 10 } });

}
},
{ connection, batches: { size: 10 } },
);
```

{% hint style="info" %}
Expand All @@ -35,18 +36,22 @@ When using batches, the default is that if the processor throws an exception, **
Sometimes it is useful to just fail specific jobs in a batch, we can accomplish this by using the job's method `setAsFailed`. See how the example above can be modified to fail specific jobs:

```typescript
const worker = new WorkerPro("My Queue", async (job: JobPro) => {
const batch = job.getBatch();

for(let i=0; i<batch.length; i++) {
const worker = new WorkerPro(
'My Queue',
async (job: JobPro) => {
const batch = job.getBatch();

for (let i = 0; i < batch.length; i++) {
const batchedJob = batch[i];
try {
await doSomethingWithBatchedJob(batchedJob);
} catch(err) {
} catch (err) {
batchedJob.setAsFailed(err);
}
}
}, { connection, batches: { size: 10 } });
}
},
{ connection, batches: { size: 10 } },
);
```

Only the jobs that are `setAsFailed` will fail, the rest will be moved to complete when the processor for the batch job completes.
Expand All @@ -58,28 +63,27 @@ Batches are handled by wrapping all the jobs in a batch into a dummy job that ke
It is possible, however, to call the getBatch function in order to retrieve all the jobs that belong to a given batch.

```typescript
worker.on('completed', job => {
const batch = job.getBatch();e
});
worker.on('completed', job => {
const batch = job.getBatch();
e;
});
```

Using a global event listener you can listen to individual job events even though they may be processed in a batch:

```typescript
import { QueueEventsPro } from "@taskforcesh/bullmq-pro"
import { QueueEventsPro } from '@taskforcesh/bullmq-pro';

const queueEvents = new QueueEventsPro(queueName, { connection });
queueEvents.on('completed', (jobId, err) => {
// ...
});
```



### Limitations

Currently, all worker options can be used with the batches, however, there are some unsupported features that may be implemented in the future:

* [Dynamic rate limit](https://docs.bullmq.io/guide/rate-limiting#manual-rate-limit)
* [Manually processing jobs](https://docs.bullmq.io/patterns/manually-fetching-jobs)
* [Dynamically delay jobs](https://docs.bullmq.io/patterns/process-step-jobs#delaying).
- [Dynamic rate limit](https://docs.bullmq.io/guide/rate-limiting#manual-rate-limit)
- [Manually processing jobs](https://docs.bullmq.io/patterns/manually-fetching-jobs)
- [Dynamically delay jobs](https://docs.bullmq.io/patterns/process-step-jobs#delaying).
9 changes: 7 additions & 2 deletions docs/gitbook/bullmq-pro/groups/pausing-groups.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

BullMQ Pro supports pausing groups globally. A group is paused when no workers will pick up any jobs that belongs to the paused group. When you pause a group, the workers that are currently busy processing a job from that group, will continue working on that job until it completes (or failed), and then will just keep idling until the group has been resumed.

Pausing a group is performed by calling the _**pauseGroup**_ method on a [queue](https://api.bullmq.pro/classes/Queue.html#pauseGroup) instance:
Pausing a group is performed by calling the _**pauseGroup**_ method on a [queue](https://api.bullmq.pro/classes/v6.Queue.html#pauseGroup) instance:

```typescript
await myQueue.pauseGroup('groupId');
Expand All @@ -16,7 +16,7 @@ Even if the groupId does not exist at that time, the groupId will be added in ou
It will return false if the group is already paused.
{% endhint %}

Resuming a group is performed by calling the _**resumeGroup**_ method on a [queue](https://api.bullmq.pro/classes/Queue.html#resumeGroup) instance:
Resuming a group is performed by calling the _**resumeGroup**_ method on a [queue](https://api.bullmq.pro/classes/v6.Queue.html#resumeGroup) instance:

```typescript
await myQueue.resumeGroup('groupId');
Expand All @@ -25,3 +25,8 @@ await myQueue.resumeGroup('groupId');
{% hint style="warning" %}
It will return false if the group does not exist or when the group is already resumed.
{% endhint %}

## Read more:

- 💡 [Pause Group API Reference](https://api.bullmq.pro/classes/v6.Queue.html#pauseGroup)
- 💡 [Resume Group API Reference](https://api.bullmq.pro/classes/v6.Queue.html#resumeGroup)
18 changes: 13 additions & 5 deletions docs/gitbook/bullmq-pro/groups/prioritized.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,22 @@
BullMQ Pro supports priorities per group. A job is prioritized in a group when group and priority options are provided together.

```typescript
await myQueue.add('paint', { foo: 'bar' }, {
await myQueue.add(
'paint',
{ foo: 'bar' },
{
group: {
id: 'groupId',
priority: 10
}
});
id: 'groupId',
priority: 10,
},
},
);
```

{% hint style="info" %}
Standard prioritized jobs get more priority than grouped prioritized jobs
{% endhint %}

## Read more:

- 💡 [Add Job API Reference](https://api.bullmq.pro/classes/v6.Queue.html#add)
1 change: 0 additions & 1 deletion docs/gitbook/bullmq-pro/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,3 @@ WORKDIR /app
ADD .npmrc /app/.npmr
```

4 changes: 2 additions & 2 deletions docs/gitbook/bullmq-pro/nestjs/producers.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,5 +66,5 @@ const job = await this.fooFlowProducer.add({
- 💡 [Queues Technique](https://docs.nestjs.com/techniques/queues)
- 💡 [Inject Queue API Reference](https://nestjs.bullmq.pro/functions/InjectQueue.html)
- 💡 [Inject Flow Producer API Reference](https://nestjs.bullmq.pro/functions/InjectFlowProducer.html)
- 💡 [QueuePro API Reference](https://api.bullmq.pro/classes/Queue.html)
- 💡 [FlowProducerPro API Reference](https://api.bullmq.pro/classes/FlowProducer.html)
- 💡 [QueuePro API Reference](https://api.bullmq.pro/classes/v6.Queue.html)
- 💡 [FlowProducerPro API Reference](https://api.bullmq.pro/classes/v6.FlowProducer.html)

0 comments on commit 235ff47

Please sign in to comment.