-
Notifications
You must be signed in to change notification settings - Fork 354
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
GitBook: [#110] docs: update queues section links
- Loading branch information
1 parent
86a1874
commit 73ac8c7
Showing
8 changed files
with
148 additions
and
155 deletions.
There are no files selected for viewing
Binary file added
BIN
+18.3 KB
docs/gitbook/.gitbook/assets/image (1) (1) (1) (1) (1) (1) (1) (2) (1).png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,114 @@ | ||
--- | ||
description: This is a basic guide to get your first queue working. | ||
--- | ||
|
||
# Quick Start | ||
|
||
## Install | ||
|
||
Install using npm: | ||
|
||
``` | ||
$ npm install bullmq | ||
``` | ||
|
||
Install using yarn: | ||
|
||
``` | ||
$ yarn add bullmq | ||
``` | ||
|
||
{% hint style="info" %} | ||
BullMQ is written in TypeScript, and although it can be used in vanilla JavaScript, all examples in this guide will be written in TypeScript. | ||
{% endhint %} | ||
|
||
Import into your project and add some jobs: | ||
|
||
```typescript | ||
import { Queue } from 'bullmq'; | ||
|
||
const myQueue = new Queue('foo'); | ||
|
||
async function addJobs() { | ||
await myQueue.add('myJobName', { foo: 'bar' }); | ||
await myQueue.add('myJobName', { qux: 'baz' }); | ||
} | ||
|
||
await addJobs(); | ||
``` | ||
|
||
{% hint style="danger" %} | ||
You need to have a Redis service running in your local computer to run these examples successfully. You can read more about Redis connections [here](guide/connections.md). | ||
{% endhint %} | ||
|
||
Jobs are added to the queue and can be processed at any time, with at least one Node.js process running a worker: | ||
|
||
```typescript | ||
import { Worker } from 'bullmq'; | ||
|
||
const worker = new Worker(queueName, async job => { | ||
// Will print { foo: 'bar'} for the first job | ||
// and { qux: 'baz' } for the second. | ||
console.log(job.data); | ||
}); | ||
``` | ||
|
||
{% hint style="info" %} | ||
You can have as many worker processes as you want, BullMQ will distribute the jobs across your workers in a round robin fashion. | ||
{% endhint %} | ||
|
||
You can listen to completed (or failed) jobs by attaching listeners to the workers: | ||
|
||
```typescript | ||
worker.on('completed', job => { | ||
console.log(`${job.id} has completed!`); | ||
}); | ||
|
||
worker.on('failed', (job, err) => { | ||
console.log(`${job.id} has failed with ${err.message}`); | ||
}); | ||
``` | ||
|
||
{% hint style="info" %} | ||
There are many other events available, check the [Guide](guide/events.md) or the [API reference](broken-reference/) for more information. | ||
{% endhint %} | ||
|
||
Sometimes you need to listen to all the workers events in a given place, for this you need to use a special class `QueueEvents`: | ||
|
||
```typescript | ||
import { QueueEvents } from 'bullmq'; | ||
|
||
const queueEvents = new QueueEvents(); | ||
|
||
queueEvents.on('waiting', ({ jobId }) => { | ||
console.log(`A job with ID ${jobId} is waiting`); | ||
}); | ||
|
||
queueEvents.on('active', ({ jobId, prev }) => { | ||
console.log(`Job ${jobId} is now active; previous status was ${prev}`); | ||
}); | ||
|
||
queueEvents.on('completed', ({ jobId, returnvalue }) => { | ||
console.log(`${jobId} has completed and returned ${returnvalue}`); | ||
}); | ||
|
||
queueEvents.on('failed', ({ jobId, failedReason }) => { | ||
console.log(`${jobId} has failed with reason ${failedReason}`); | ||
}); | ||
``` | ||
|
||
You may also access the timestamp of the event, which looks like "1580456039332-0". | ||
|
||
```typescript | ||
import { QueueEvents } from 'bullmq'; | ||
|
||
const queueEvents = new QueueEvents(); | ||
|
||
queueEvents.on('progress', ({ jobId, data }, timestamp) => { | ||
console.log(`${jobId} reported progress ${data} at ${timestamp}`); | ||
}); | ||
``` | ||
|
||
{% hint style="danger" %} | ||
For performance reasons, the events emited by a `QueueEvents` instance do not contain the `Job` instance, only the `jobId`. Use the `Job#fromId` method if you need the `Job` instance. | ||
{% endhint %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,114 +1,42 @@ | ||
--- | ||
description: This is a basic guide to get your first queue working. | ||
description: General description of BullMQ and its features | ||
--- | ||
|
||
# Quick Start | ||
# What is BullMQ | ||
|
||
## Install | ||
BullMQ is a [Node.js](https://nodejs.org) library that implements a fast and robust queue system built on top of [Redis](https://redis.io) that helps in resolving many modern age micro-services architectures. | ||
|
||
Install using npm: | ||
The library is designed so that it will fulfil the following goals: | ||
|
||
``` | ||
$ npm install bullmq | ||
``` | ||
* Exactly once queue semantics, i.e., attempts to deliver every message exactly one time, but it will deliver at least once in the worst case scenario\*. | ||
* Easy to scale horizontally. Add more workers for processing jobs in parallel. | ||
* Consistent. | ||
* High performant. Try to get the highest possible throughput from Redis by combining efficient .lua scripts and pipelining. | ||
|
||
Install using yarn: | ||
View the repository, see open issues, and contribute back [on GitHub](https://github.com/taskforcesh/bullmq)! | ||
|
||
``` | ||
$ yarn add bullmq | ||
``` | ||
## **Features** | ||
|
||
{% hint style="info" %} | ||
BullMQ is written in TypeScript, and although it can be used in vanilla JavaScript, all examples in this guide will be written in TypeScript. | ||
{% endhint %} | ||
If you are new to Message Queues, you may wonder why they are needed after all. Queues can solve many different problems in an elegant way, from smoothing out processing peaks to creating robust communication channels between micro-services or offloading heavy work from one server to many smaller workers, and many other use cases. Check the [Patterns](patterns/producer-consumer.md) section for getting some inspiration and information about best practices. | ||
|
||
Import into your project and add some jobs: | ||
* [x] **Minimal CPU usage due to a polling-free design** | ||
* [x] **Distributed job execution based on Redis** | ||
* [x] **LIFO and FIFO jobs** | ||
* [x] **Priorities** | ||
* [x] **Delayed jobs** | ||
* [x] **Scheduled and repeatable jobs according to cron specifications** | ||
* [x] **Retries of failed jobs** | ||
* [x] **Concurrency setting per worker** | ||
* [x] **Threaded (sandboxed) processing functions** | ||
* [x] **Automatic recovery from process crashes** | ||
|
||
```typescript | ||
import { Queue } from 'bullmq'; | ||
### Used by | ||
|
||
const myQueue = new Queue('foo'); | ||
BullMQ is used by many organizations big and small, here are some notable examples: | ||
|
||
async function addJobs() { | ||
await myQueue.add('myJobName', { foo: 'bar' }); | ||
await myQueue.add('myJobName', { qux: 'baz' }); | ||
} | ||
![](.gitbook/assets/clipart1565701.png) | ||
|
||
await addJobs(); | ||
``` | ||
![](.gitbook/assets/wordmark-logo.png) | ||
|
||
{% hint style="danger" %} | ||
You need to have a Redis service running in your local computer to run these examples successfully. You can read more about Redis connections [here](guide/connections.md). | ||
{% endhint %} | ||
![](.gitbook/assets/datawrapper-logo.png) | ||
|
||
Jobs are added to the queue and can be processed at any time, with at least one Node.js process running a worker: | ||
|
||
```typescript | ||
import { Worker } from 'bullmq'; | ||
|
||
const worker = new Worker(queueName, async job => { | ||
// Will print { foo: 'bar'} for the first job | ||
// and { qux: 'baz' } for the second. | ||
console.log(job.data); | ||
}); | ||
``` | ||
|
||
{% hint style="info" %} | ||
You can have as many worker processes as you want, BullMQ will distribute the jobs across your workers in a round robin fashion. | ||
{% endhint %} | ||
|
||
You can listen to completed (or failed) jobs by attaching listeners to the workers: | ||
|
||
```typescript | ||
worker.on('completed', job => { | ||
console.log(`${job.id} has completed!`); | ||
}); | ||
|
||
worker.on('failed', (job, err) => { | ||
console.log(`${job.id} has failed with ${err.message}`); | ||
}); | ||
``` | ||
|
||
{% hint style="info" %} | ||
There are many other events available, check the [Guide](guide/events.md) or the [API reference](broken-reference/) for more information. | ||
{% endhint %} | ||
|
||
Sometimes you need to listen to all the workers events in a given place, for this you need to use a special class `QueueEvents`: | ||
|
||
```typescript | ||
import { QueueEvents } from 'bullmq'; | ||
|
||
const queueEvents = new QueueEvents(); | ||
|
||
queueEvents.on('waiting', ({ jobId }) => { | ||
console.log(`A job with ID ${jobId} is waiting`); | ||
}); | ||
|
||
queueEvents.on('active', ({ jobId, prev }) => { | ||
console.log(`Job ${jobId} is now active; previous status was ${prev}`); | ||
}); | ||
|
||
queueEvents.on('completed', ({ jobId, returnvalue }) => { | ||
console.log(`${jobId} has completed and returned ${returnvalue}`); | ||
}); | ||
|
||
queueEvents.on('failed', ({ jobId, failedReason }) => { | ||
console.log(`${jobId} has failed with reason ${failedReason}`); | ||
}); | ||
``` | ||
|
||
You may also access the timestamp of the event, which looks like "1580456039332-0". | ||
|
||
```typescript | ||
import { QueueEvents } from 'bullmq'; | ||
|
||
const queueEvents = new QueueEvents(); | ||
|
||
queueEvents.on('progress', ({ jobId, data }, timestamp) => { | ||
console.log(`${jobId} reported progress ${data} at ${timestamp}`); | ||
}); | ||
``` | ||
|
||
{% hint style="danger" %} | ||
For performance reasons, the events emited by a `QueueEvents` instance do not contain the `Job` instance, only the `jobId`. Use the `Job#fromId` method if you need the `Job` instance. | ||
{% endhint %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.