"Pending, in queue" message remain even after bulk action - update of product attributes is done

[MrPotatox]

 - Summary of the issue,

After attempting to update product attributes in bulk I have a "stuck" system message of
[Task "Update attributes for 679 selected products": 0 item(s) have been scheduled for update.]
This has been in "Pending, in queue" status for about 2 weeks now.

Preconditions

Magento ver. 2.3.5-p2, 2.4-develop
Using Magento Cloud Commerce (Valid for Magento Open Source as well)

Steps to reproduce

1. In Admin navigate to Catalog -> Products page.
2. Select all products.
3. In drop-down 'Actions' menu select "Update attributes"
4. Change desired value of any attribute (like description) and save.

Expected result

After all product attributes changed to the desired value, bulk action message should inform about task complete

Actual result

Bulk actions / system message still displaying at top of page.

[m2-assistant]

Hi @MrPotatox. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[VladimirZaets]

@magento I am working on it

[jackrevate]

Having same issue with 2.4.0, upgraded from 2.3.5-p1 to 2.3.5-p2 then to 2.4.0.

It's really annoying, hoping this can be fixed asap.

[codemastering]

Having the same issue with 2.4.0, upgraded from 2.3.5 to 2.4.0, Thankfully, I took back up before upgrade to 2.4.0, Restored back to 2.3.5

Looking forward to any temporary fix for this issue.

Thanks

[perfpcs]

I am having this same problem with 3 different servers running M 2.4.0. CE All have stuck tasks that used to process in minutes sitting in the bulk task grid and notices for weeks. Using rabbitmq for message broker. So is someone working on this issue @VladimirZaets ? I can't see how anyone on M 2.4.0 is not having this problem. I would be happy even with a temporary fix if anyone has any ideas on this. How could M 2.4.0 even have been released with such a bad bug?? Thanks in advance if anyone can help.

[perfpcs]

To add further info for us it is various consumers and tasks getting stuck and not just the mass product attribute updates. For us we have 50-100 tasks involving Nosto extension and their product updates that are done via cron jobs:

Task "Sync 1500 Nosto products": 15 item(s) have been scheduled for update.

These seem to complete too because I have seen the number of tasks from 50-165 just for this Nosto task. So because the number goes up and down I have to think some of these are processing but then they are not updating status in the grid. I never see any of them "completed" like I used to see in previous Magento versions where I would dismiss the completed task notices. I have never had no unfinished tasks where the tasks are completed fairly fast and none show pending like is the case for 90% of the time in previous Magento versions (you see nothing pending for tasks in bulk actions grid).

I am curious @codemastering and @jackrevate are you guys using mysql or rabbitmq messaging and are you on Cloud Commerce or CE?

[TonMarton]

Hi, I am using mysql and CE (2.4) and having the exact same issue.

For me all the updates were completed (I changed price on 10 products only). But the system messages are stuck.

I don't know if it is helpful or not, but my queue_message table has the following contents:
| id | topic_name | body | | 10 | product_action_attribute.update | {"id":null,"bulk_uuid":"95891348-0c1b-478e-9d69-8e36c356977d","topic_name":"product_action_attribute.update","serialized_data":"{\"meta_information\":\"Update product attributes\",\"product_ids\":[\"3\",\"4\",\"5\",\"7\",\"8\",\"9\",\"16\",\"17\",\"18\",\"19\"],\"store_id\":0,\"website_id\":\"0\",\"attributes\":{\"price\":\"99999\"}}","result_serialized_data":null,"status":4,"result_message":null,"error_code":null} |

[perfpcs]

@TonMarton I think the more people that post this problem the better chance we have of Magento fixing it. But its appearing to be a widespread problem.

[perfpcs]

The more I look at this the more it appears that the bulk jobs are not actually stuck but are actually done and complete but the notice of completion is not getting back to the bulk task grid and therefore it never updates all these "pendings" to "completes". So it seems to be a communication issue to tell the bulk task notices that they are complete. You never see any bulk tasks listed as "complete" in M 2.4.0 like i used to see in M 2.3.5. Then I would click the "Dismiss all the completed tasks" link. It never gets to this state to dismiss any of them as the tasks are not updating that they are complete. So that is what the problem is as I see it.
Anyone have any ideas how to resolve? Anyone having these same issues with M 2.4.0?

If anyone from team Magento feels I need to rather start a new post or topic on this I can since what I am experiencing is not exactly the same as the OP. Just let me know.

[codemastering]

100% agreed with @perfpcs

i am facing same issue with Magento 2.4

anyone, please provide the fix for this.

[HamishBlank]

Also seeing this issue in Magento 2.4, job has been carried out but notification is still being displayed as "Pending, in queue..."

[perfpcs]

@VladimirZaets ..any update to this issue since it appears to be widespead Magento bug?

[m2-assistant]

Hi @engcom-Oscar. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

✅ Confirmed by @engcom-Oscar
Thank you for verifying the issue. Based on the provided information internal tickets MC-37717 were created

Issue Available: @engcom-Oscar, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[forcecodema]

@gwharton I changed the line in /magento_root/vendor/magento/module-asynchronous-operations/etc/db_schema.xml then run bin/magento setup:upgrade, but nothing changed...

[gwharton]

The change probably worked, its just that it only allows you to acknowledge new bulk tasks. I think existing bulk tasks that are stuck need to be manually cleared from the database unfortunately. Check and clear ut the offending items from magento_bulk, and hopefully new ones will work properly now.

[forcecodema]

I cleared all the messages. And when the new showed up i tried to dismiss them. Operation clears/acknowledges them in database (if you refresh the page they are gone), but the ui returns "Attention Something went wrong". It looks like the problem is with AJAX call?

/admin_xxxxxx/bulk/notification/dismiss/key/39e1eb81366237663820b4026c18b2a14d0e433c039549b46f4da305e2ea7d22/?isAjax=true

It returns an empty response - no errors in system.log

[forcecodema]

I found solution - the problem was JSON response in Dismiss.php action - I fixed it modifying the file as follows:

<?php
/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */
namespace Magento\AsynchronousOperations\Controller\Adminhtml\Notification;

use Magento\AsynchronousOperations\Model\BulkNotificationManagement;
use Magento\Backend\App\Action\Context;
use Magento\Backend\App\Action;
use Magento\Framework\Controller\Result\JsonFactory;

/**
 * Class Bulk Notification Dismiss Controller
 */
class Dismiss extends Action
{
    /**
     * @var BulkNotificationManagement
     */
    private $notificationManagement;

    /**
     * @var JsonFactory
     */
    protected $resultJsonFactory;

    /**
     * Class constructor.
     *
     * @param Context $context
     * @param BulkNotificationManagement $notificationManagement
     * @param JsonFactory $resultJsonFactory
     */
    public function __construct(
        Context $context,
        BulkNotificationManagement $notificationManagement,
        JsonFactory $resultJsonFactory
    ) {
        parent::__construct($context);
        $this->notificationManagement = $notificationManagement;
        $this->resultJsonFactory = $resultJsonFactory;
    }

    /**
     * @inheritDoc
     */
    protected function _isAllowed()
    {
        return $this->_authorization->isAllowed('Magento_Logging::system_magento_logging_bulk_operations');
    }

    /**
     * {@inheritdoc}
     */
    public function execute()
    {
        $bulkUuids = [];
        foreach ((array)$this->getRequest()->getParam('uuid', []) as $bulkUuid) {
            $bulkUuids[] = (string)$bulkUuid;
        }

        $isAcknowledged = $this->notificationManagement->acknowledgeBulks($bulkUuids);

        /** @var \Magento\Framework\Controller\Result\Json $result */
        $result = $this->resultJsonFactory->create(ResultFactory::TYPE_JSON);
        $response = new \Magento\Framework\DataObject();
        if (!$isAcknowledged) {
            $result->setHttpResponseCode(400);
            $response->setError(true);
        } else {
            $result->setHttpResponseCode(200);
            $response->setSuccess(true);
        }

        $result->setData($response);

        return $result;
    }
}

[bhaveshpp]

After update, Magento 2.3.4 to Magento 2.4 and I was facing this issue.
Now the issue is solved after the update: https://github.com/magento/magento2/pull/29814/files
Thank you @gwharton @nuzil

[bibz0r]

How can we apply this fix?

[gaiterjones]

@bibz0r for Magento 2.4.x in vendor/magento/module-asynchronous-operations/etc/db_schema.xml change

<column xsi:type="int" name="operation_key" padding="10" unsigned="true" nullable="false"

to

<column xsi:type="int" name="operation_key" padding="10" unsigned="true" nullable="true"

then run bin/magento s:up

Old jobs might well still be stuck but new jobs should complete.

[davirs]

I have same situation here with magento 2.4.0.

I did the fix in Dismiss.php as mentionated above and I did the fix inside db_schema.xml.
After did this procidures I did a setup upgrade, I cleared cache, I forced cron:run... but the problem persists.

Someone can help me to clear this queue?:

there is some more thing to do to solve this problem? Thank you

[in-session]

The problem still exists even on Magento 2.4.1 with PHP 7.4

The cron is running so far, even starting it via SSH did not help.

Mentioned fix does not solve the problem: https://github.com/magento/magento2/pull/29814/files

[OvalMedia]

The fix works as the bulk actions get executed via cron. But the message queue in magento_bulk does not get cleared.

[sumeetmagento]

I have the same issue in magento 2.3.5-p1, as per the fix https://github.com/magento/magento2/pull/29814/files, such column not exists in magento 2.3.5, How to fix that ?

Are you able to solve it because I am also getting in 2.3.5 ?

[sumeetmagento]

This problem also arises in Magento 2.3.5
Does anyone know how to solve this?

Cron is working fine I cross-check with indexing and when run this command - php bin/magento queue:consumers:start product_action_attribute.update then the problem is solved but it's not a good way, again and again, to run these commands

[sumeetmagento]

I have the same issue in magento 2.3.5-p1, as per the fix https://github.com/magento/magento2/pull/29814/files, such column not exists in magento 2.3.5, How to fix that ?

@magecoders
are you able to solve on Magento 2.3.5 ?

[JustADude-86]

@bibz0r for Magento 2.4.x in vendor/magento/module-asynchronous-operations/etc/db_schema.xml change

<column xsi:type="int" name="operation_key" padding="10" unsigned="true" nullable="false"

to

<column xsi:type="int" name="operation_key" padding="10" unsigned="true" nullable="true"

then run bin/magento s:up

Old jobs might well still be stuck but new jobs should complete.

For me, this solved this issue on 2.4.1. Existing "stuck" jobs I had manually removed from the database..but all "new bulk changes" after fix run without any problems.

[in-session]

In my case it seems like there were problems with the cron after the update of magento.

After reinstall the cron bin/magento cron:install --force removed old jobs manually in the DB and add the fix https://github.com/magento/magento2/pull/29814/files, it works fine

[Eddcapone]

I have this issue with Magento 2.4.2 Enterprise. I also tested with a fresh magento.

[Eddcapone]

@magento give me 2.4-develop instance

[magento-deployment-service]

Hi @Eddcapone. Thank you for your request. I'm working on Magento instance for you.

[magento-deployment-service]

Hi @Eddcapone, here is your Magento Instance: https://4ac6ac9a19ad3ca16429aeb14bf67949-2-4-develop.instances.magento-community.engineering
Admin access: https://4ac6ac9a19ad3ca16429aeb14bf67949-2-4-develop.instances.magento-community.engineering/admin_1213
Login: 3c3791ca
Password: f9890c8ab87c

[Eddcapone]

I updated the weight attribute of some products, lets keep an eye to it:
https://4ac6ac9a19ad3ca16429aeb14bf67949-2-4-develop.instances.magento-community.engineering/admin_1213/bulk/bulk/details/uuid/679e50be-a03c-40ac-9569-75300cf2c6aa/

UPDATE: It finished successfully on the develop instance.

Why does it not work on a fresh magento 2.4.2 enterprise then, even not after calling cron:run manually?

[Eddcapone]

I figured out that the jobs are suddenly running successfully if you execute php bin/magento queue:consumer:start product_action_attribute.update. So it looks like the cron job fails to execute this command for unknown reasons

[tuyennn]

Applied this patch #29814 on magento cloud 2.4.0-p1
Database changed
MariaDB [main]> EXPLAIN magento_operation;
+------------------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| operation_key | int(10) unsigned | YES | | NULL | |
| bulk_uuid | varbinary(39) | YES | MUL | NULL | |
| topic_name | varchar(255) | YES | | NULL | |
| serialized_data | blob | YES | | NULL | |
| result_serialized_data | blob | YES | | NULL | |
| status | smallint(6) | YES | | 0 | |
| error_code | smallint(6) | YES | | NULL | |
| result_message | varchar(255) | YES | | NULL | |
+------------------------+------------------+------+-----+---------+----------------+
9 rows in set (0.00 sec)

It's actually nothing was fixed

[davirs]

Hi guys. I solved this problem editing the file env.php. I don't know why my magento had this parameters:

'cron_consumers_runner' => [ 'cron_run' => true, 'max_messages' => 20000, 'consumers' => [ 'consumer1', 'consumer2' ] ]

I removed this code and all catalog updates worked pretty.

I hope it helps somebody

[ueltonVeslei]

Hi guys. I solved this problem editing the file env.php. I don't know why my magento had this parameters:

'cron_consumers_runner' => [ 'cron_run' => true, 'max_messages' => 20000, 'consumers' => [ 'consumer1', 'consumer2' ] ]

I removed this code and all catalog updates worked pretty.

I hope it helps somebody

Worked for me, thx buddy

[trojson]

I got the same issue today on 2.4.2, removed the jobs from DB - table bulk back to normal, not sure what happen on it but I think probably about the cron jobs incorrectly, also catalog price rule cron job issue once applied.

[tuyennn]

269b47a
1d9e07b
#31495

[toweringmedia]

php bin/magento queue:consumer:start product_action_attribute.update
This worked for me thanks davirs!

[BernardRobbins]

php bin/magento queue:consumer:start product_action_attribute.update

M2.4.6-p3
RabbitMQ 3.11.20

This worked for me.

I had queued messages in RabbitMQ that were stuck after doing bulk product updates.