Skip to content

Commit

Permalink
GitBook: [#110] docs: update queues section links
Browse files Browse the repository at this point in the history
  • Loading branch information
roggervalf authored and gitbook-bot committed Apr 22, 2022
1 parent 86a1874 commit 73ac8c7
Show file tree
Hide file tree
Showing 8 changed files with 148 additions and 155 deletions.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
114 changes: 114 additions & 0 deletions docs/gitbook/README (1).md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
description: This is a basic guide to get your first queue working.
---

# Quick Start

## Install

Install using npm:

```
$ npm install bullmq
```

Install using yarn:

```
$ yarn add bullmq
```

{% hint style="info" %}
BullMQ is written in TypeScript, and although it can be used in vanilla JavaScript, all examples in this guide will be written in TypeScript.
{% endhint %}

Import into your project and add some jobs:

```typescript
import { Queue } from 'bullmq';

const myQueue = new Queue('foo');

async function addJobs() {
await myQueue.add('myJobName', { foo: 'bar' });
await myQueue.add('myJobName', { qux: 'baz' });
}

await addJobs();
```

{% hint style="danger" %}
You need to have a Redis service running in your local computer to run these examples successfully. You can read more about Redis connections [here](guide/connections.md).
{% endhint %}

Jobs are added to the queue and can be processed at any time, with at least one Node.js process running a worker:

```typescript
import { Worker } from 'bullmq';

const worker = new Worker(queueName, async job => {
// Will print { foo: 'bar'} for the first job
// and { qux: 'baz' } for the second.
console.log(job.data);
});
```

{% hint style="info" %}
You can have as many worker processes as you want, BullMQ will distribute the jobs across your workers in a round robin fashion.
{% endhint %}

You can listen to completed (or failed) jobs by attaching listeners to the workers:

```typescript
worker.on('completed', job => {
console.log(`${job.id} has completed!`);
});

worker.on('failed', (job, err) => {
console.log(`${job.id} has failed with ${err.message}`);
});
```

{% hint style="info" %}
There are many other events available, check the [Guide](guide/events.md) or the [API reference](broken-reference/) for more information.
{% endhint %}

Sometimes you need to listen to all the workers events in a given place, for this you need to use a special class `QueueEvents`:

```typescript
import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents();

queueEvents.on('waiting', ({ jobId }) => {
console.log(`A job with ID ${jobId} is waiting`);
});

queueEvents.on('active', ({ jobId, prev }) => {
console.log(`Job ${jobId} is now active; previous status was ${prev}`);
});

queueEvents.on('completed', ({ jobId, returnvalue }) => {
console.log(`${jobId} has completed and returned ${returnvalue}`);
});

queueEvents.on('failed', ({ jobId, failedReason }) => {
console.log(`${jobId} has failed with reason ${failedReason}`);
});
```

You may also access the timestamp of the event, which looks like "1580456039332-0".

```typescript
import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents();

queueEvents.on('progress', ({ jobId, data }, timestamp) => {
console.log(`${jobId} reported progress ${data} at ${timestamp}`);
});
```

{% hint style="danger" %}
For performance reasons, the events emited by a `QueueEvents` instance do not contain the `Job` instance, only the `jobId`. Use the `Job#fromId` method if you need the `Job` instance.
{% endhint %}
124 changes: 26 additions & 98 deletions docs/gitbook/README.md
Original file line number Diff line number Diff line change
@@ -1,114 +1,42 @@
---
description: This is a basic guide to get your first queue working.
description: General description of BullMQ and its features
---

# Quick Start
# What is BullMQ

## Install
BullMQ is a [Node.js](https://nodejs.org) library that implements a fast and robust queue system built on top of [Redis](https://redis.io) that helps in resolving many modern age micro-services architectures.

Install using npm:
The library is designed so that it will fulfil the following goals:

```
$ npm install bullmq
```
* Exactly once queue semantics, i.e., attempts to deliver every message exactly one time, but it will deliver at least once in the worst case scenario\*.
* Easy to scale horizontally. Add more workers for processing jobs in parallel.
* Consistent.
* High performant. Try to get the highest possible throughput from Redis by combining efficient .lua scripts and pipelining.

Install using yarn:
View the repository, see open issues, and contribute back [on GitHub](https://github.com/taskforcesh/bullmq)!

```
$ yarn add bullmq
```
## **Features**

{% hint style="info" %}
BullMQ is written in TypeScript, and although it can be used in vanilla JavaScript, all examples in this guide will be written in TypeScript.
{% endhint %}
If you are new to Message Queues, you may wonder why they are needed after all. Queues can solve many different problems in an elegant way, from smoothing out processing peaks to creating robust communication channels between micro-services or offloading heavy work from one server to many smaller workers, and many other use cases. Check the [Patterns](patterns/producer-consumer.md) section for getting some inspiration and information about best practices.

Import into your project and add some jobs:
* [x] **Minimal CPU usage due to a polling-free design**
* [x] **Distributed job execution based on Redis**
* [x] **LIFO and FIFO jobs**
* [x] **Priorities**
* [x] **Delayed jobs**
* [x] **Scheduled and repeatable jobs according to cron specifications**
* [x] **Retries of failed jobs**
* [x] **Concurrency setting per worker**
* [x] **Threaded (sandboxed) processing functions**
* [x] **Automatic recovery from process crashes**

```typescript
import { Queue } from 'bullmq';
### Used by

const myQueue = new Queue('foo');
BullMQ is used by many organizations big and small, here are some notable examples:

async function addJobs() {
await myQueue.add('myJobName', { foo: 'bar' });
await myQueue.add('myJobName', { qux: 'baz' });
}
![](.gitbook/assets/clipart1565701.png)

await addJobs();
```
![](.gitbook/assets/wordmark-logo.png)

{% hint style="danger" %}
You need to have a Redis service running in your local computer to run these examples successfully. You can read more about Redis connections [here](guide/connections.md).
{% endhint %}
![](.gitbook/assets/datawrapper-logo.png)

Jobs are added to the queue and can be processed at any time, with at least one Node.js process running a worker:

```typescript
import { Worker } from 'bullmq';

const worker = new Worker(queueName, async job => {
// Will print { foo: 'bar'} for the first job
// and { qux: 'baz' } for the second.
console.log(job.data);
});
```

{% hint style="info" %}
You can have as many worker processes as you want, BullMQ will distribute the jobs across your workers in a round robin fashion.
{% endhint %}

You can listen to completed (or failed) jobs by attaching listeners to the workers:

```typescript
worker.on('completed', job => {
console.log(`${job.id} has completed!`);
});

worker.on('failed', (job, err) => {
console.log(`${job.id} has failed with ${err.message}`);
});
```

{% hint style="info" %}
There are many other events available, check the [Guide](guide/events.md) or the [API reference](broken-reference/) for more information.
{% endhint %}

Sometimes you need to listen to all the workers events in a given place, for this you need to use a special class `QueueEvents`:

```typescript
import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents();

queueEvents.on('waiting', ({ jobId }) => {
console.log(`A job with ID ${jobId} is waiting`);
});

queueEvents.on('active', ({ jobId, prev }) => {
console.log(`Job ${jobId} is now active; previous status was ${prev}`);
});

queueEvents.on('completed', ({ jobId, returnvalue }) => {
console.log(`${jobId} has completed and returned ${returnvalue}`);
});

queueEvents.on('failed', ({ jobId, failedReason }) => {
console.log(`${jobId} has failed with reason ${failedReason}`);
});
```

You may also access the timestamp of the event, which looks like "1580456039332-0".

```typescript
import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents();

queueEvents.on('progress', ({ jobId, data }, timestamp) => {
console.log(`${jobId} reported progress ${data} at ${timestamp}`);
});
```

{% hint style="danger" %}
For performance reasons, the events emited by a `QueueEvents` instance do not contain the `Job` instance, only the `jobId`. Use the `Job#fromId` method if you need the `Job` instance.
{% endhint %}
8 changes: 4 additions & 4 deletions docs/gitbook/SUMMARY.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Table of contents

* [What is BullMQ](what-is-bullmq.md)
* [Quick Start](README.md)
* [API Reference](https://api.docs.bullmq.io/)
* [What is BullMQ](README.md)
* [Quick Start](<README (1).md>)
* [API Reference](https://api.docs.bullmq.io)
* [Changelog](changelog.md)

## Guide
Expand Down Expand Up @@ -61,7 +61,7 @@
* [Groups](bullmq-pro/groups/README.md)
* [Rate limiting](bullmq-pro/groups/rate-limiting.md)
* [Concurrency](bullmq-pro/groups/concurrency.md)
* [API Reference](https://api.bullmq.pro/)
* [API Reference](https://api.bullmq.pro)

## Bull

Expand Down
7 changes: 0 additions & 7 deletions docs/gitbook/bull/patterns/returning-job-completions.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,3 @@
---
description: >-
A common pattern is where you have a cluster of queue processors that just
process jobs as fast as they can, and some other services that need to take
the result of this processors and do something wi
---

# Returning Job Completions

A common pattern is where you have a cluster of queue processors that just process jobs as fast as they can, and some other services that need to take the result of these processors and do something with it, maybe storing results in a database.
Expand Down
2 changes: 1 addition & 1 deletion docs/gitbook/guide/introduction.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Introduction

BullMQ is based in 5 classes that together can be used to resolve many different problems. These classes are [_**Queue**_](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.queue.md), [_**Worker**_](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.worker.md), [_**QueueScheduler**_](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.queuescheduler.md), [_**QueueEvents**_](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.queueevents.md) and [_**FlowProducer**_](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.flowproducer.md).
BullMQ is based in 5 classes that together can be used to resolve many different problems. These classes are [_**Queue**_](https://api.docs.bullmq.io/classes/Queue.html), [_**Worker**_](https://api.docs.bullmq.io/classes/Worker.html), [_**QueueScheduler**_](https://api.docs.bullmq.io/classes/QueueScheduler.html), [_**QueueEvents**_](https://api.docs.bullmq.io/classes/QueueEvents.html) and [_**FlowProducer**_](https://api.docs.bullmq.io/classes/FlowProducer.html).

The first class you should know about is the _Queue_ class. This class represents a queue and can be used for adding _**jobs**_ to the queue as well as some other basic manipulation such as pausing, cleaning or getting data from the queue.

Expand Down
6 changes: 3 additions & 3 deletions docs/gitbook/guide/queues/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ See [Connections](../connections.md) for details on how to pass Redis details to

When you instance a Queue, BullMQ will just _upsert_ a small "meta-key", so if the queue existed before it will just pick it up and you can continue adding jobs to it.

The most important method is probably the [_**add**_](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.queue.add.md) method. This method allows you to add jobs to the queue in different fashions:
The most important method is probably the [_**add**_](https://api.docs.bullmq.io/classes/Queue.html#add) method. This method allows you to add jobs to the queue in different fashions:

```typescript
await queue.add('paint', { colour: 'red' });
Expand All @@ -28,7 +28,7 @@ When adding a job you can also specify an options object. This options object ca
await queue.add('paint', { colour: 'blue' }, { delay: 5000 });
```

The job will now wait **at** **least** 5 seconds before it is processed.&#x20;
The job will now wait **at** **least** 5 seconds before it is processed.

{% hint style="danger" %}
In order for delay jobs to work you need to have at least one _QueueScheduler_ somewhere in your infrastructure. Read more [here](../queuescheduler.md).
Expand All @@ -38,4 +38,4 @@ There are many other options available such as priorities, backoff settings, lif

## Read more:

- 💡 [Queue API Reference](https://github.com/taskforcesh/bullmq/blob/master/docs/gitbook/api/bullmq.queue.md)
* 💡 [Queue API Reference](https://api.docs.bullmq.io/classes/Queue.html)
42 changes: 0 additions & 42 deletions docs/gitbook/what-is-bullmq.md

This file was deleted.

0 comments on commit 73ac8c7

Please sign in to comment.