Catalog product list widget sort order configured in backend does not match in frontend

[BorisovskiP]

emphasized text<!---
Please review our guidelines before adding a new issue: https://github.com/magento/magento2/wiki/Issue-reporting-guidelines
Fields marked with (*) are required. Please don't remove the template.
--]
When configuring products in Catalog->Category->Products in Category in the admin, the same products are not sorted by their configured order.

When setVisibility is called (Magento\CatalogWidget\Block\Product\ProductsList::createCollection() line 308), it sets cat*index.category*id=3.

Generated query is as followed:

 SELECT DISTINCT  `e`.*, `cat*index`.`position` AS `cat_index_position`, `price_index`.`price`, `price_index`.`tax_class_id`, `price_index`.`final_price`, IF(price_index.tier_price IS NOT NULL, LEAST(price_index.min_price, price_index.tier_price), price_index.min_price) AS `minimal_price`, `price_index`.`min_price`, `price_index`.`max_price`, `price_index`.`tier_price` FROM `catalog_product_entity` AS `e` INNER JOIN `catalog_category_product_index_store2` AS `cat_index` ON cat_index.product_id=e.entity_id AND cat_index.store_id=2 AND cat_index.visibility IN(2, 4) AND cat_index.category_id=3 INNER JOIN `catalog_product_index_price` AS `price_index` ON price_index.entity_id = e.entity_id AND price_index.website_id = '1' AND price_index.customer_group_id = 0 WHERE (((IFNULL(`e`.`entity_id`, 0) IN (SELECT `catalog_category_product`.`product_id` FROM `catalog_category_product` WHERE (category_id IN ('5244')))) )) ORDER BY `e`.`created*at` desc

Preconditions (*)

[!---
Provide the exact Magento version (example: 2.3.2) and any important information on the environment where bug is reproducible.
--]

1. Magento 2.3.4
2. PHP 7.2

Steps to reproduce (*)

[!---
Important: Provide a set of clear steps to reproduce this bug. We can not provide support without clear instructions on how to reproduce.
-->

1. Create category ( Catalog -> Categories -> Add Root Category)

2. Rename category to TestCategory

3. Add 3 or more products to Category (Catalog-> Products -> Select/Edit -> Categories -> TestCategory -> Done -> Save)

4. Open Catalog -> Categories -> TestCategory -> Products in Category

5. Set position on productX to 1, Set position on productY to 2 , Set position on productZ to 3 and Save

6. Add CMS Page (Content -> Page -> Add New Page -> PageTitle = TestPage)

7. Set Content on TestPage to the following (change the category ID; template can be chosen freely)
   widget type="Magento\CatalogWidget\Block\Product\ProductsList" show*pager="0" products_count="10" template="Magento_CatalogWidget::product/widget/content/home-grid.phtml" **conditions_encoded="<sup>[{{1:[type:Magento](|CatalogWidget||Model||Rule||Condition||Combine,aggregator:all,value:1,new_child:``^],1--1:[`type`:`Magento||CatalogWidget||Model||Rule||Condition||Product`,`attribute`:`category*ids`,`operator`:`==`,`value`:`1`^])"}}

8. Insert widget with widget type - Catalog Product List and condition - Category is
   

Expected result (*)

1. productZ (position3) is first, productX (position 1) is second, productY (position 2) is last

[m2-assistant]

Hi @BorisovskiP. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

@BorisovskiP do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[m2-assistant]

Hi @shikhamis11. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[shikhamis11]

@magento give me 2.4-develop instance

[magento-engcom-team]

Hi @shikhamis11. Thank you for your request. I'm working on Magento 2.4-develop instance for you

[magento-engcom-team]

Hi @shikhamis11, here is your Magento instance.
Admin access: https://i-27126-2-4-develop.instances.magento-community.engineering/admin_8734
Login: ca6ddb22 Password: 040f68dcd4a6
Instance will be terminated in up to 3 hours.

[BorisovskiP]

@shikhamis11 Can you tell me whats the status of this issue?

[shikhamis11]

hello @BorisovskiP
I am able to reproduce this issue but I find that there is no toolbar available on the CMS pages with the widget so not able to decide what will be default sorting

[BorisovskiP]

hello @shikhamis11,
Do you have any ideas how you would proceed?

[shikhamis11]

I will discuss this issue with @sdzhepa and update the progress here

[BorisovskiP]

Any updates on the issue @shikhamis11 ?

[sdzhepa]

@engcom-Alfa , @engcom-Echo

Could you please help with the investigation and confirmation process for this issue?

[m2-assistant]

Hi @engcom-Alfa. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[BorisovskiP]

@engcom-Alfa Any updates on this issue?

[engcom-Alfa]

Hi @BorisovskiP. Sorry for the delay.

We've checked the issue and can conclude that indeed, the sort order by position has not been preserved

Actual Result:
productZ (position 3) is first, productY (position 2) is second and productX (position 1) is last

But we are not quite sure how sorting should be set for widgets.
Perhaps a better solution to this problem would be to add sort selection functionality when creating the Catalog product list widget.
Like that:

@BorisovskiP What do you think about it?
Thanks!

[Stefan0x]

@engcom-Alfa
I think this would be a great solution!

[BorisovskiP]

@engcom-Alfa I think that by default, sort by position should be applied, but having the possibility to set the sort order in the widget does sound nice.

[BorisovskiP]

@engcom-Alfa what is the current status of this issue? Do you have any updates?

[BorisovskiP]

@engcom-Alfa Any news regarding the issue?

[Edelstahlshop]

@shikhamis11 @magento-engcom-team : Is there any update? Thanks for your efforts until. I would really like to get this issue fixed.

[BorisovskiP]

@engcom-Hotel is this issue fixed? I tried to open the PR you referenced in #27126 (comment) but it does not exist.

[engcom-Hotel]

Hello @BorisovskiP,

I think this is due to some access required. I can see that the PR is merged.

Thanks

[BorisovskiP]

Hello @engcom-Hotel, could you tell me in which version has the PR been merged? If its not in 2.4.4, could you please point me to the place where the changes have been made so that we can apply a patch to the current version of Magento that we use that still has the issue present?

[engcom-Hotel]

Hello @BorisovskiP,

We are re-checking the previous activities on this, will update u soon.

Thanks

[m2-assistant]

Hi @engcom-November. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-November]

Verified the issue on Magento 2.4-develop branch and the issue is still reproducible.
Sort by Position / or id is not working and there is no sort by drop down is also present for user to select. Hence considering this as feature request. Updating the description of manual steps to reproduce.

[engcom-November]

Hi @BorisovskiP ,
We rechecked the issue again after internal discussion and confirmation from PO, this is an expected behavior but not a bug. Please find the findings below:
Under page builder:
we came across that user has 3 options to select products by. when you select Category, the default sorting behavior is Position and it works fine.
If you select Condition, the default sorting behavior is Newest Products First (entity_id desc) and if you look closer there is no Position sorting option in this case.

With normal installation:
In the widget in normal installation for CMS page when we add the condition the default behavior is same as of with the page builder . The Newest Products First shows up first.



We can back this situation with further clarification and example:
If we add the product list widget with a condition where we will use multiple categories , sorting by position becomes ambiguous as we don't know which positions to sort by. the product can have different positions in different categories.This makes sense because Condition allows random criteria that user can define to select products by which may or may not include category condition. And sorting by position does not work properly in this case because the current sorting implementation does take into account category ids used in the condition.

To conclude the same: we observe that as under page builder the default behaviour for condition is newest first, the same should be applied under CMS page widget option which makes us suggest that this should be the expected behaviour and the one valid.

Resulting we are closing this issue. Kindly reopen/create new issue if you are still facing any issues.
Thank you.

[mobweb]

Where can we open a feature request for this? Right now there seems to be no way to specify an order for products shown in a "Catalog Products List" widget. But I strongly suggest adding this, as a merchant might want to show selected products on a CMS page with a specific order, e.g. the most important product first.