What is queue

docs/book/v1/overview.md

@@ -1,6 +1,11 @@
 # Overview
 
-> Dotkernel component used to queue tasks to be processed asynchronously based on [netglue/laminas-messenger](https://github.com/netglue/laminas-messenger)
+> [Dotkernel Queue](https://github.com/dotkernel/dot-queue) is a component based on [**Symfony Messenger**](https://github.com/symfony/messenger) that is used to queue asynchronous tasks.
+[netglue/laminas-messenger](https://github.com/netglue/laminas-messenger) is a layer that turns Symfony Messenger into a middleware compatible with Mezzio/Laminas applications.
+
+Some everyday **operations are time-consuming and resource-intensive**, so it's best if they run on separate machines, decoupled from the regular request-response cycle.
+**Asynchronous execution** performed by background workers ensures that these operations won't overload the main platform.
+It allows the main platform to return a response and remain responsive for new requests, while tasks with long execution times are scheduled to run later.
 
 ![Queue process](https://docs.dotkernel.org/img/queue/schema.png)
 

docs/book/v1/what-is-queue.md

@@ -0,0 +1,40 @@
+# Tasks Delegated to the Queue
+
+Normally, the tasks delegated to the queue:
+
+- Take extended periods of time to execute and may be interrupted by PHP limitations (like the PHP `max_execution_time` parameter).
+- Are external which introduces delays from authentication, awaiting responses from potentially multiple interrogations, or may even fail completely because the server is offline.
+- Are not part of the PHP response (sending emails, processing videos).
+
+Here are some example tasks meant for the queue:
+
+- **Data Processing** like big data analytics, scientific simulations, or mathematical computations.
+- **File Handling & Media Processing** like video and image processing, or compression/decompression of large files.
+- **Networking** like uploading/downloading large files.
+- **Database Operations** like imports/exports or migrations.
+- **System & Infrastructure Tasks** like OS updates, software compilation, CI pipelines.
+
+The long execution times are caused by:
+
+- Data size - gigabytes, terabytes etc.
+- Complex algorithms.
+- Hardware limitations - CPU speed, memory, storage, network bandwidth.
+- External dependencies - waiting on APIs or human input.
+
+## How the Queue Works
+
+- The queue system has an active **daemon** that listens for TPC connections on a specific port and **stores incoming messages** into Redis.
+This method supports a **large number of requests per second** without overloading.
+- Operations are then **scheduled for execution** when resources are available.
+The order of the execution uses the **FIFO** (First-In, First-Out) method where the oldest request is processed first, followed by newer requests.
+
+## Main Features
+
+- **Logging** - Ensures maintainability and allows debugging.
+- **Security** - The **firewall** allows requests only from whitelisted IPs for fast response times.
+- **Retry Mechanism** - Retries failing tasks a certain number of times before removing a task from the queue.
+- **Reporting** - The logs enable developers to investigate various metrics via console commands like:
+    - Queue length - How many jobs are in the to-do list.
+    - Processing time per job.
+    - Error rates - How many messages failed.
+    - Throughput - Jobs/sec processed.

mkdocs.yml

@@ -9,6 +9,7 @@ nav:
   - Home: index.md
   - v1:
       - Overview: v1/overview.md
+      - "Understanding the Queue": v1/what-is-queue.md
       - Server Setup: v1/server-setup.md
       - Installation: v1/installation.md
       - Control Commands: v1/control-commands.md