From 8dc02595b595c9db8ab23e3591af53d9e38fc782 Mon Sep 17 00:00:00 2001 From: Bruno Roeder Date: Fri, 4 Oct 2019 11:55:21 -0300 Subject: [PATCH 01/12] insert Manage message queues instructions to enable bulk --- guides/v2.3/rest/bulk-endpoints.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/guides/v2.3/rest/bulk-endpoints.md b/guides/v2.3/rest/bulk-endpoints.md index ca0e35cddb2..216a02f2b39 100644 --- a/guides/v2.3/rest/bulk-endpoints.md +++ b/guides/v2.3/rest/bulk-endpoints.md @@ -10,7 +10,9 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing. +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configuring the message queues'. For more details see: [ +Manage message queues +]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). ### Routes From 4b3c96430dc9ee8b1683b9e80cb21ecbcc4ac097 Mon Sep 17 00:00:00 2001 From: Bruno Roeder Date: Fri, 4 Oct 2019 11:59:45 -0300 Subject: [PATCH 02/12] insert Manage message queues instructions to enable bulk --- guides/v2.2/rest/bulk-endpoints.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index a32edc2fa86..2b073cb21b2 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -11,7 +11,9 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing. +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configuring the message queues'. For more details see: [ +Manage message queues +]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). ### Routes From 912e028e58fbb68820a1bfaa70283976f06c0a9f Mon Sep 17 00:00:00 2001 From: Bruno Roeder Date: Fri, 4 Oct 2019 12:05:38 -0300 Subject: [PATCH 03/12] fix typo --- guides/v2.2/rest/bulk-endpoints.md | 2 +- guides/v2.3/rest/bulk-endpoints.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index 2b073cb21b2..7cdfb0ac655 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -11,7 +11,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configuring the message queues'. For more details see: [ +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues'. For more details see: [ Manage message queues ]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). diff --git a/guides/v2.3/rest/bulk-endpoints.md b/guides/v2.3/rest/bulk-endpoints.md index 216a02f2b39..50f41b40fb6 100644 --- a/guides/v2.3/rest/bulk-endpoints.md +++ b/guides/v2.3/rest/bulk-endpoints.md @@ -10,7 +10,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configuring the message queues'. For more details see: [ +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues'. For more details see: [ Manage message queues ]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). From 04466454610fc34569c5e37a054a96635ca12391 Mon Sep 17 00:00:00 2001 From: Bruno Roeder Date: Fri, 4 Oct 2019 12:06:17 -0300 Subject: [PATCH 04/12] fix typo --- guides/v2.2/rest/bulk-endpoints.md | 2 +- guides/v2.3/rest/bulk-endpoints.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index 7cdfb0ac655..3143f0c4deb 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -11,7 +11,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues'. For more details see: [ +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues. For more details see: [ Manage message queues ]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). diff --git a/guides/v2.3/rest/bulk-endpoints.md b/guides/v2.3/rest/bulk-endpoints.md index 50f41b40fb6..51cd63f32c8 100644 --- a/guides/v2.3/rest/bulk-endpoints.md +++ b/guides/v2.3/rest/bulk-endpoints.md @@ -10,7 +10,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues'. For more details see: [ +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues. For more details see: [ Manage message queues ]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). From 45bf9a3bfaa1c4f459b26045da7ae969ed6509bd Mon Sep 17 00:00:00 2001 From: Bruno Roeder Date: Fri, 4 Oct 2019 14:15:33 -0300 Subject: [PATCH 05/12] warnings and tips hot to configure message queues processingguides/v2.2/rest/bulk-endpoints.md --- guides/v2.3/rest/bulk-endpoints.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/guides/v2.3/rest/bulk-endpoints.md b/guides/v2.3/rest/bulk-endpoints.md index 51cd63f32c8..2d7b77e08ce 100644 --- a/guides/v2.3/rest/bulk-endpoints.md +++ b/guides/v2.3/rest/bulk-endpoints.md @@ -9,10 +9,14 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. +{: .bs-callout .bs-callout-warning} +By default, Magento uses RabbitMQ (AMQP) for processing message queues. Before using Bulk operations, you should see [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). + +{:.bs-callout .bs-callout-info} +Alternatively you can use CRON for processing message queues without RabbitMQ installed, see [Manage message queues]({{ page.baseurl }}/config-guide/mq/manage-message-queues.html). + {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues. For more details see: [ -Manage message queues -]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing. ### Routes From 342b6bc3d76fb725e63dfe7fb694fd42ebb40b42 Mon Sep 17 00:00:00 2001 From: Donald Booth Date: Wed, 16 Oct 2019 15:58:06 -0500 Subject: [PATCH 06/12] formatting --- guides/v2.2/rest/bulk-endpoints.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index 3143f0c4deb..6020e9a6b92 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -11,9 +11,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues. For more details see: [ -Manage message queues -]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). +Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues. For more details see: [Manage message queues]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). ### Routes From cf64456869c68e59bfebf993f7d9c412479b93c1 Mon Sep 17 00:00:00 2001 From: Margaret Eker Date: Mon, 28 Oct 2019 22:52:16 -0500 Subject: [PATCH 07/12] Updated notes and links for message queue configuration --- .../v2.2/install-gde/prereq/install-rabbitmq.md | 6 +++--- guides/v2.2/rest/bulk-endpoints.md | 4 ++-- .../v2.3/install-gde/prereq/install-rabbitmq.md | 17 +++++++++-------- guides/v2.3/rest/bulk-endpoints.md | 10 ++-------- 4 files changed, 16 insertions(+), 21 deletions(-) diff --git a/guides/v2.2/install-gde/prereq/install-rabbitmq.md b/guides/v2.2/install-gde/prereq/install-rabbitmq.md index ca84868638f..d5f17373c7c 100644 --- a/guides/v2.2/install-gde/prereq/install-rabbitmq.md +++ b/guides/v2.2/install-gde/prereq/install-rabbitmq.md @@ -24,8 +24,8 @@ The message queue system must be established before you install Magento. The bas 1. Install RabbitMQ and any prerequisites. 1. Connect RabbitMQ and Magento. -{:.bs-callout .bs-callout-info} -A basic message queue system can be implemented on {{site.data.var.ee}} using cron instead of RabbitMQ. See [Configure message queues]({{ page.baseurl }}/config-guide/mq/manage-mysql.html) for more information. +{:.bs-callout-info} +You can use MySQL or RabbitMQ for message queue processing. For details on setting up the message queue system, see [Message queues overview]({{ page.baseurl }}/extension-dev-guide/message-queues/message-queues.html). If you are using the Bulk API with {{ site.data.var.ee }} v2.3.3 or later, the default message queue configuration uses RabbitMQ as the message broker. ## Install RabbitMQ on Ubuntu {#ubuntu-install} @@ -133,7 +133,7 @@ To configure support for SSL, edit the `ssl` and `ssl_options` parameters in the ## Start the message queue consumers -After you have connected {{site.data.var.ee}} and RabbitMQ, you must start the message queue consumers. See [Configure message queues]({{ page.baseurl }}/config-guide/mq/manage-mysql.html) for details. +After you have connected {{site.data.var.ee}} and RabbitMQ, you must start the message queue consumers. See [Configure message queues]({{page.baseurl}}/config-guide/mq/manage-message-queues.html#start-message-queue-consumers) for details. {:.ref-header} Related topics diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index 6020e9a6b92..971390ed94a 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -10,8 +10,8 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. -{:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing, after configure the message queues. For more details see: [Manage message queues]({{ page.baseurl }}config-guide/mq/manage-message-queues.html). +{:.bs-callout-info} +Use the [`bin/magento queue:consumers:start async.operations.all`](https://devdocs.magento.com/guides/v2.3/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). ### Routes diff --git a/guides/v2.3/install-gde/prereq/install-rabbitmq.md b/guides/v2.3/install-gde/prereq/install-rabbitmq.md index 4ef741c7482..3f9d90bc555 100644 --- a/guides/v2.3/install-gde/prereq/install-rabbitmq.md +++ b/guides/v2.3/install-gde/prereq/install-rabbitmq.md @@ -21,7 +21,7 @@ The message queue system must be established before you install Magento. The bas 1. Connect RabbitMQ and Magento. {:.bs-callout .bs-callout-info"} -A basic message queue system can be implemented using cron instead of RabbitMQ. See [Configure message queues]({{page.baseurl}}/config-guide/mq/manage-mysql.html) for more information. +You can use MySQL or RabbitMQ for message queue processing. For details on setting up the message queue system, see [Message queues overview]({{ page.baseurl }}/extension-dev-guide/message-queues/message-queues.html). If you are using the Bulk API with {{ site.data.var.ee }} v2.3.3 or later, the message queue system configuration defaults to using RabbitMQ as the message broker. See [Start message queue consumers]({{page.baseurl}}/config-guide/mq/manage-message-queues.html#start-message-queue-consumers) for more information. ## Install RabbitMQ on Ubuntu {#ubuntu-install} @@ -63,15 +63,16 @@ rpm -Uvh erlang-solutions-1.0-1.noarch.rpm The RabbitMQ server is included on CentOS, but the version is often old. RabbitMQ recommends installing the package from their website. 1. Download [rabbitmq-server-3.5.6-1.noarch.rpm](https://www.rabbitmq.com/releases/rabbitmq-server/v3.5.6/rabbitmq-server-3.5.6-1.noarch.rpm){:target="_blank"}. + 1. Run the following commands as a user with root permissions: - ```bash - rpm --import https://www.rabbitmq.com/rabbitmq-signing-key-public.asc - ``` + ```bash + rpm --import https://www.rabbitmq.com/rabbitmq-signing-key-public.asc + ``` - ```bash - yum install rabbitmq-server-3.5.6-1.noarch.rpm - ``` + ```bash + yum install rabbitmq-server-3.5.6-1.noarch.rpm + ``` Refer to [Installing on RPM-based Linux](https://www.rabbitmq.com/install-rpm.html){:target="_blank"} for more information. @@ -148,7 +149,7 @@ To configure support for SSL, edit the `ssl` and `ssl_options` parameters in the ## Start the message queue consumers -After you have connected Magento and RabbitMQ, you must start the message queue consumers. See [Configure message queues]({{page.baseurl}}/config-guide/mq/manage-mysql.html) for details. +After you have connected {{site.data.var.ee}} and RabbitMQ, you must start the message queue consumers. See [Configure message queues]({{page.baseurl}}/config-guide/mq/manage-message-queues.html#start-message-queue-consumers) for details. {:.ref-header} Related topics diff --git a/guides/v2.3/rest/bulk-endpoints.md b/guides/v2.3/rest/bulk-endpoints.md index ec0ef8756a7..07011007f72 100644 --- a/guides/v2.3/rest/bulk-endpoints.md +++ b/guides/v2.3/rest/bulk-endpoints.md @@ -9,14 +9,8 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. -{: .bs-callout .bs-callout-warning} -By default, Magento uses RabbitMQ (AMQP) for processing message queues. Before using Bulk operations, you should see [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). - -{:.bs-callout .bs-callout-info} -Alternatively you can use CRON for processing message queues without RabbitMQ installed, see [Manage message queues]({{ page.baseurl }}/config-guide/mq/manage-message-queues.html). - -{:.bs-callout .bs-callout-tip} -Use the `bin/magento queue:consumers:start async.operations.all` command to enable bulk endpoint processing. +{:.bs-callout-info} +Use the [`bin/magento queue:consumers:start async.operations.all`](https://devdocs.magento.com/guides/v2.3/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). ### Routes From cf0dfa98bfe1350bb8706e9987857ed547072551 Mon Sep 17 00:00:00 2001 From: Margaret Eker Date: Mon, 28 Oct 2019 23:29:24 -0500 Subject: [PATCH 08/12] Added prerequisite for RabbitMQ install --- .../rest/tutorials/configurable-product/config-product-intro.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/guides/v2.2/rest/tutorials/configurable-product/config-product-intro.md b/guides/v2.2/rest/tutorials/configurable-product/config-product-intro.md index 5ab81c37170..0c4e6307f44 100644 --- a/guides/v2.2/rest/tutorials/configurable-product/config-product-intro.md +++ b/guides/v2.2/rest/tutorials/configurable-product/config-product-intro.md @@ -25,6 +25,8 @@ This **5-step tutorial** generally takes **45 minutes**. * Install a REST client. You can use any REST client to send calls to Magento. [Postman](https://www.getpostman.com/) is recommended. +* [Install and configure RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html), which is the default message broker for bulk API endpoints. + * Obtain an admin authorization token. All calls in this tutorial require administrator privileges. See [Generate the admin token]({{ page.baseurl }}/rest/tutorials/prerequisite-tasks/create-admin-token.html) for more information. ### Other resources From 4b908acd518f19a6dd47dfc746a82b74fb55e423 Mon Sep 17 00:00:00 2001 From: Margaret Eker Date: Thu, 31 Oct 2019 09:41:25 -0500 Subject: [PATCH 09/12] - Fixed hard-coded link - Removed confusing statement about managing message queues using cron jobs - Added note and link about configuring consumers_runner job on Cloud --- .../v2.2/config-guide/mq/manage-message-queues.md | 9 ++++++--- guides/v2.2/install-gde/prereq/install-rabbitmq.md | 2 +- guides/v2.2/rest/bulk-endpoints.md | 2 +- .../v2.3/config-guide/mq/manage-message-queues.md | 13 ++++++++----- guides/v2.3/install-gde/prereq/install-rabbitmq.md | 2 +- guides/v2.3/rest/bulk-endpoints.md | 2 +- 6 files changed, 18 insertions(+), 12 deletions(-) diff --git a/guides/v2.2/config-guide/mq/manage-message-queues.md b/guides/v2.2/config-guide/mq/manage-message-queues.md index c30293bb88a..ea8f0a24ec6 100644 --- a/guides/v2.2/config-guide/mq/manage-message-queues.md +++ b/guides/v2.2/config-guide/mq/manage-message-queues.md @@ -9,15 +9,15 @@ functional_areas: - Setup --- -If you don't want to implement the RabbitMQ solution, you can manage message queues with cron jobs (or an external process manager) and the command line to ensure that consumers are retrieving messages. +You can manage message queues from the command line using cron jobs or an external process manager to ensure that consumers are retrieving messages. ## Process management Cron jobs are the default mechanism to restart consumers. Processes started by `cron` consume the specified number of messages and then terminate. Re-running `cron` restarts the consumer. -The following shows a `crontab` configuration for running consumers in our implementation, it is the example for understanding how it works: +The following example shows the Magento `crontab` configuration for running consumers: -*/app/code/Magento/MessageQueue/etc/crontab.xml* +> /app/code/Magento/MessageQueue/etc/crontab.xml ```xml ... @@ -40,6 +40,9 @@ You can also use a process manager such as [Supervisor](http://supervisord.org/i * Cron job `consumers_runner` runs all defined consumers * Each consumer processes 10000 messages and then terminates +{:.bs-callout-info} +If your {{ site.data.var.ee }} store is hosted on the Cloud platform, use the [`CRON_CONSUMERS_RUNNER`]({{ page.baseurl }}/cloud/env/variables-deploy.html#cron_consumers_runner) to configure the `consumers_runner` cron job. + #### Specific configuration Edit */app/etc/env.php* file for configure cron job `consumers_runner` diff --git a/guides/v2.2/install-gde/prereq/install-rabbitmq.md b/guides/v2.2/install-gde/prereq/install-rabbitmq.md index d5f17373c7c..3347ba6f1f4 100644 --- a/guides/v2.2/install-gde/prereq/install-rabbitmq.md +++ b/guides/v2.2/install-gde/prereq/install-rabbitmq.md @@ -25,7 +25,7 @@ The message queue system must be established before you install Magento. The bas 1. Connect RabbitMQ and Magento. {:.bs-callout-info} -You can use MySQL or RabbitMQ for message queue processing. For details on setting up the message queue system, see [Message queues overview]({{ page.baseurl }}/extension-dev-guide/message-queues/message-queues.html). If you are using the Bulk API with {{ site.data.var.ee }} v2.3.3 or later, the default message queue configuration uses RabbitMQ as the message broker. +You can use MySQL or RabbitMQ for message queue processing. For details on setting up the message queue system, see [Message queues overview]({{ page.baseurl }}/extension-dev-guide/message-queues/message-queues.html). If you are using the Bulk API with {{ site.data.var.ee }}, the message queue system configuration defaults to using RabbitMQ as the message broker. See [Start message queue consumers]({{page.baseurl}}/config-guide/mq/manage-message-queues.html#start-message-queue-consumers) for more information. ## Install RabbitMQ on Ubuntu {#ubuntu-install} diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index 971390ed94a..39969fda385 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -11,7 +11,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout-info} -Use the [`bin/magento queue:consumers:start async.operations.all`](https://devdocs.magento.com/guides/v2.3/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). +Use the [`bin/magento queue:consumers:start async.operations.all`]({{ page.baseurl }}/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). ### Routes diff --git a/guides/v2.3/config-guide/mq/manage-message-queues.md b/guides/v2.3/config-guide/mq/manage-message-queues.md index d08b2e6d29c..04c9979a83a 100644 --- a/guides/v2.3/config-guide/mq/manage-message-queues.md +++ b/guides/v2.3/config-guide/mq/manage-message-queues.md @@ -9,15 +9,15 @@ functional_areas: - Setup --- -If you don't want to implement the RabbitMQ solution, you can manage message queues with cron jobs (or an external process manager) and the command line to ensure that consumers are retrieving messages. +You can manage message queues from the command line using cron jobs or an external process manager to ensure that consumers are retrieving messages. ## Process management Cron jobs are the default mechanism to restart consumers. Processes started by `cron` consume the specified number of messages and then terminate. Re-running `cron` restarts the consumer. -The following shows a `crontab` configuration for running consumers in our implementation, it is the example for understanding how it works: +The following example shows the Magento `crontab` configuration for running consumers: -*/app/code/Magento/MessageQueue/etc/crontab.xml* +> /app/code/Magento/MessageQueue/etc/crontab.xml ```xml ... @@ -27,7 +27,7 @@ The following shows a `crontab` configuration for running consumers in our imple ... ``` -{:.bs-callout .bs-callout-info} +{:.bs-callout-info} How often you check message queues depends on your business logic and available system resources. In general, you'll probably want to check for newly created customers and send welcome emails more frequently than a more resource intensive process (e.g., updating your catalog). You should define `cron` schedules according to your business needs.

It can be configured in Admin Panel **Stores > Settings > Configuration > Advanced > System > Cron configuration options for group: consumers**

See [Configure and run cron]({{ page.baseurl }}/config-guide/cli/config-cli-subcommands-cron.html) for more information about using `cron` with Magento. You can also use a process manager such as [Supervisor](http://supervisord.org/index.html) to monitor the status of processes. The manager can use the command line to restart the processes as needed. @@ -40,9 +40,12 @@ You can also use a process manager such as [Supervisor](http://supervisord.org/i * Cron job `consumers_runner` runs all defined consumers * Each consumer processes 10000 messages and then terminates +{:.bs-callout-info} +If your {{ site.data.var.ee }} store is hosted on the Cloud platform, use the [`CRON_CONSUMERS_RUNNER`]({{ page.baseurl }}/cloud/env/variables-deploy.html#cron_consumers_runner) to configure the `consumers_runner` cron job. + #### Specific configuration -Edit */app/etc/env.php* file for configure cron job `consumers_runner` +Edit the `/app/etc/env.php` file to configure the cron job `consumers_runner`. ```php ... diff --git a/guides/v2.3/install-gde/prereq/install-rabbitmq.md b/guides/v2.3/install-gde/prereq/install-rabbitmq.md index 3f9d90bc555..2c827949bba 100644 --- a/guides/v2.3/install-gde/prereq/install-rabbitmq.md +++ b/guides/v2.3/install-gde/prereq/install-rabbitmq.md @@ -21,7 +21,7 @@ The message queue system must be established before you install Magento. The bas 1. Connect RabbitMQ and Magento. {:.bs-callout .bs-callout-info"} -You can use MySQL or RabbitMQ for message queue processing. For details on setting up the message queue system, see [Message queues overview]({{ page.baseurl }}/extension-dev-guide/message-queues/message-queues.html). If you are using the Bulk API with {{ site.data.var.ee }} v2.3.3 or later, the message queue system configuration defaults to using RabbitMQ as the message broker. See [Start message queue consumers]({{page.baseurl}}/config-guide/mq/manage-message-queues.html#start-message-queue-consumers) for more information. +You can use MySQL or RabbitMQ for message queue processing. For details on setting up the message queue system, see [Message queues overview]({{ page.baseurl }}/extension-dev-guide/message-queues/message-queues.html). If you are using the Bulk API with {{ site.data.var.ee }}, the message queue system configuration defaults to using RabbitMQ as the message broker. See [Start message queue consumers]({{page.baseurl}}/config-guide/mq/manage-message-queues.html#start-message-queue-consumers) for more information. ## Install RabbitMQ on Ubuntu {#ubuntu-install} diff --git a/guides/v2.3/rest/bulk-endpoints.md b/guides/v2.3/rest/bulk-endpoints.md index 07011007f72..3f1b0f92230 100644 --- a/guides/v2.3/rest/bulk-endpoints.md +++ b/guides/v2.3/rest/bulk-endpoints.md @@ -10,7 +10,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout-info} -Use the [`bin/magento queue:consumers:start async.operations.all`](https://devdocs.magento.com/guides/v2.3/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). +Use the [`bin/magento queue:consumers:start async.operations.all`]({{ page.baseurl }}/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). ### Routes From 5917e65f975b118ccabca2540b45c7de0cdd7325 Mon Sep 17 00:00:00 2001 From: Margaret Eker Date: Mon, 4 Nov 2019 13:35:06 -0600 Subject: [PATCH 10/12] Update guides/v2.2/config-guide/mq/manage-message-queues.md --- guides/v2.2/config-guide/mq/manage-message-queues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guides/v2.2/config-guide/mq/manage-message-queues.md b/guides/v2.2/config-guide/mq/manage-message-queues.md index ea8f0a24ec6..4c1c9bcf5d3 100644 --- a/guides/v2.2/config-guide/mq/manage-message-queues.md +++ b/guides/v2.2/config-guide/mq/manage-message-queues.md @@ -41,7 +41,7 @@ You can also use a process manager such as [Supervisor](http://supervisord.org/i * Each consumer processes 10000 messages and then terminates {:.bs-callout-info} -If your {{ site.data.var.ee }} store is hosted on the Cloud platform, use the [`CRON_CONSUMERS_RUNNER`]({{ page.baseurl }}/cloud/env/variables-deploy.html#cron_consumers_runner) to configure the `consumers_runner` cron job. +If your {{ site.data.var.ee }} store is hosted on the Cloud platform, use the [`CRON_CONSUMERS_RUNNER`]({{ page.baseurl }}/cloud/env/variables-deploy.html#cron_consumers_runner) deploy variable to configure the `consumers_runner` cron job. #### Specific configuration From a966539331c032143deedb68f399711e2ad82acf Mon Sep 17 00:00:00 2001 From: Margaret Eker Date: Mon, 4 Nov 2019 18:18:52 -0600 Subject: [PATCH 11/12] Fixed broken link Changed _Start consumers_ topic title to _Start message queue consumers_ for consistency with v2.3 --- guides/v2.2/config-guide/mq/manage-message-queues.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guides/v2.2/config-guide/mq/manage-message-queues.md b/guides/v2.2/config-guide/mq/manage-message-queues.md index 4c1c9bcf5d3..8aa1d0d348b 100644 --- a/guides/v2.2/config-guide/mq/manage-message-queues.md +++ b/guides/v2.2/config-guide/mq/manage-message-queues.md @@ -66,7 +66,7 @@ Edit */app/etc/env.php* file for configure cron job `consumers_runner` ## Command line interface -### Start consumers +### Start message queue consumers Use the `magento` command to start message queue consumers. You can start multiple consumers simultaneously. From ce9fd32e3cf9c485057b581f4d3ac8dbf60c00a2 Mon Sep 17 00:00:00 2001 From: Margaret Eker Date: Wed, 6 Nov 2019 13:08:37 -0600 Subject: [PATCH 12/12] Removed unneeded link reference --- guides/v2.2/rest/bulk-endpoints.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guides/v2.2/rest/bulk-endpoints.md b/guides/v2.2/rest/bulk-endpoints.md index 39969fda385..8a3e6cb864f 100644 --- a/guides/v2.2/rest/bulk-endpoints.md +++ b/guides/v2.2/rest/bulk-endpoints.md @@ -11,7 +11,7 @@ functional_areas: Bulk API endpoints differ from other REST endpoints in that they combine multiple calls of the same type into an array and execute them as a single request. The endpoint handler splits the array into individual entities and writes them as separate messages to the message queue. {:.bs-callout-info} -Use the [`bin/magento queue:consumers:start async.operations.all`]({{ page.baseurl }}/config-guide/cli/config-cli-subcommands-queue.html) command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). +Use the `bin/magento queue:consumers:start async.operations.all` command to start the consumer that handles asynchronous and bulk API messages. Also, before using the Bulk API to process messages, you must install and configure RabbitMQ, which is the default message broker. See [RabbitMQ]({{ page.baseurl }}/install-gde/prereq/install-rabbitmq.html). ### Routes