New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Categories Don't Display All Products #8681

Closed
paulrinaldi1 opened this Issue Feb 26, 2017 · 26 comments

Comments

Projects
None yet
@paulrinaldi1

paulrinaldi1 commented Feb 26, 2017

2.1.4

When clicking my categories on my homepage not all products are shown.

For example if a category has 2000 products, when displaying on the site it shows a max of lets say 500.

This causes all sorts of problems with filtering etc because not all products are being included.

To be clear if a category contains 2000 products as an example, and I go on my site and click that category, the range will say "Showing 1-32 products out of 5xx" or some other number, it does not show the full amount of products. It isn't just a display issue either.

@gputignano

This comment has been minimized.

Show comment
Hide comment
@gputignano

gputignano Feb 26, 2017

I also get the same problem after migrating from magento 1.9.x version to magento v2.1.x.

gputignano commented Feb 26, 2017

I also get the same problem after migrating from magento 1.9.x version to magento v2.1.x.

@hubaig

This comment has been minimized.

Show comment
Hide comment
@hubaig

hubaig Feb 26, 2017

I have the same problem.

My categories have products in the hundreds but the category only shows a few. I have tried many things such as reindexing and clearing cache.

Not sure how to resolve this issue as I do not see any exception or errors.

My Index looks fine:

[-x-x-x-x-x-x@s01 public_html]$ ./bin/magento indexer:status
Design Config Grid: Ready
Customer Grid: Ready
Category Products: Ready
Product Categories: Ready
Product Price: Ready
Product EAV: Ready
Stock: Ready
Catalog Rule Product: Ready
Catalog Product Rule: Ready
Catalog Search: Ready

Where should I be looking to figure out the problem. Any help would be appreciated.

hubaig commented Feb 26, 2017

I have the same problem.

My categories have products in the hundreds but the category only shows a few. I have tried many things such as reindexing and clearing cache.

Not sure how to resolve this issue as I do not see any exception or errors.

My Index looks fine:

[-x-x-x-x-x-x@s01 public_html]$ ./bin/magento indexer:status
Design Config Grid: Ready
Customer Grid: Ready
Category Products: Ready
Product Categories: Ready
Product Price: Ready
Product EAV: Ready
Stock: Ready
Catalog Rule Product: Ready
Catalog Product Rule: Ready
Catalog Search: Ready

Where should I be looking to figure out the problem. Any help would be appreciated.

@hubaig

This comment has been minimized.

Show comment
Hide comment
@hubaig

hubaig Feb 27, 2017

I have been trying many things. So I disabled all products and set them back to enabled and that fixed things.

hubaig commented Feb 27, 2017

I have been trying many things. So I disabled all products and set them back to enabled and that fixed things.

@gputignano

This comment has been minimized.

Show comment
Hide comment
@gputignano

gputignano Feb 27, 2017

I've tried to disable and than enable again but doesn't work.

gputignano commented Feb 27, 2017

I've tried to disable and than enable again but doesn't work.

@hubaig

This comment has been minimized.

Show comment
Hide comment
@hubaig

hubaig Feb 27, 2017

@gputignano After enabling and disabling them try rebuilding the index and clearing the cache. Also check the status of your reindex to make sure it completes.

Also clear the cache of your browser.

hubaig commented Feb 27, 2017

@gputignano After enabling and disabling them try rebuilding the index and clearing the cache. Also check the status of your reindex to make sure it completes.

Also clear the cache of your browser.

@EmilyPepperman

This comment has been minimized.

Show comment
Hide comment
@EmilyPepperman

EmilyPepperman Feb 27, 2017

Contributor

We had this same issue... sorta solved it... just resave the category... all the products will come back. Also reorder the products in the category... both of those actions solved the issue.

Contributor

EmilyPepperman commented Feb 27, 2017

We had this same issue... sorta solved it... just resave the category... all the products will come back. Also reorder the products in the category... both of those actions solved the issue.

@gputignano

This comment has been minimized.

Show comment
Hide comment
@gputignano

gputignano Feb 27, 2017

I tried but doesn't work for me.

gputignano commented Feb 27, 2017

I tried but doesn't work for me.

@AirmanAJK

This comment has been minimized.

Show comment
Hide comment
@AirmanAJK

AirmanAJK Mar 7, 2017

This one was hard to track down. In the following file:

\vendor\magento\module-catalog\Model\Indexer\Category\Product\AbstractAction.php

There is a constant variable called RANGE_CATEGORY_STEP, and it is set to 500. This variable is used when the function isRangingNeeded() returns true for an index (which is the default unless overridden). This effectively limits what gets put in the product category index table which in turn shows a limited result set of products on category pages.

My solution was to extend the class Magento\Catalog\Model\Indexer\Category\Product\Action\Full, overriding the function isRangingNeeded() and returning false instead of the default value of true.

I think Magento has this behavior to keep massive catalogs under control, especially when the category hierarchy has many levels and products, but this "feature" was unacceptable for my store.

AirmanAJK commented Mar 7, 2017

This one was hard to track down. In the following file:

\vendor\magento\module-catalog\Model\Indexer\Category\Product\AbstractAction.php

There is a constant variable called RANGE_CATEGORY_STEP, and it is set to 500. This variable is used when the function isRangingNeeded() returns true for an index (which is the default unless overridden). This effectively limits what gets put in the product category index table which in turn shows a limited result set of products on category pages.

My solution was to extend the class Magento\Catalog\Model\Indexer\Category\Product\Action\Full, overriding the function isRangingNeeded() and returning false instead of the default value of true.

I think Magento has this behavior to keep massive catalogs under control, especially when the category hierarchy has many levels and products, but this "feature" was unacceptable for my store.

@nfishbane

This comment has been minimized.

Show comment
Hide comment
@nfishbane

nfishbane Mar 7, 2017

We are facing the same issue after migration from 1.8 - indexing, disable/enable , cache - did not work, the only way we found to solve it is to run a script in the background that saves all products this put them back in the category but since we have a lot of products takes a long time, would love to get some other ideas how to solve it

nfishbane commented Mar 7, 2017

We are facing the same issue after migration from 1.8 - indexing, disable/enable , cache - did not work, the only way we found to solve it is to run a script in the background that saves all products this put them back in the category but since we have a lot of products takes a long time, would love to get some other ideas how to solve it

@twistedatrocity

This comment has been minimized.

Show comment
Hide comment
@twistedatrocity

twistedatrocity Mar 7, 2017

Had same issue after data migration, I think this may be more of a bug with the data migration tool. Workaround: Manually opened each category in admin and clicked save. Re-indexed afterward and all products show up on frontend category view now.

twistedatrocity commented Mar 7, 2017

Had same issue after data migration, I think this may be more of a bug with the data migration tool. Workaround: Manually opened each category in admin and clicked save. Re-indexed afterward and all products show up on frontend category view now.

@hubaig

This comment has been minimized.

Show comment
Hide comment
@hubaig

hubaig Mar 9, 2017

Is there a permanent solution to this for now?

hubaig commented Mar 9, 2017

Is there a permanent solution to this for now?

@gclift

This comment has been minimized.

Show comment
Hide comment
@gclift

gclift Mar 13, 2017

As AirmanAJK pointed out above, the RANGE_CATEGORY_STEP is causing a bug. It's using that number to create a set of select queries with a limit of RANGE_CATEGORY_STEP. Once that number has been satisfied, it's creating the next select statement but with entity Id (category id) being greater than the previous select. For example, we have 50 categories being index. Once the first select statement reaches the RANGE_CATEGORY_STEP value, it creates a new statement and adds "AND (cc.entity_id > '42')". In our case, the 500th row was in category 42 along with 12 more products. But since the second select statement included category ids greater than 42, it missed 11 products. I hope this explanation makes sense. I'm going to follow AirmanAJK recommendation and overwrite the \Magento\Catalog\Model\Indexer\Category\Product\Action\Full class and set the isRangingNeeded return value to false.

gclift commented Mar 13, 2017

As AirmanAJK pointed out above, the RANGE_CATEGORY_STEP is causing a bug. It's using that number to create a set of select queries with a limit of RANGE_CATEGORY_STEP. Once that number has been satisfied, it's creating the next select statement but with entity Id (category id) being greater than the previous select. For example, we have 50 categories being index. Once the first select statement reaches the RANGE_CATEGORY_STEP value, it creates a new statement and adds "AND (cc.entity_id > '42')". In our case, the 500th row was in category 42 along with 12 more products. But since the second select statement included category ids greater than 42, it missed 11 products. I hope this explanation makes sense. I'm going to follow AirmanAJK recommendation and overwrite the \Magento\Catalog\Model\Indexer\Category\Product\Action\Full class and set the isRangingNeeded return value to false.

@hubaig

This comment has been minimized.

Show comment
Hide comment
@hubaig

hubaig Mar 14, 2017

@AirmanAJK and @gclift. Would it be possible for you to write some steps to show how to do this? I get the idea but I am not 100% sure on what file to edit and what to exactly edit in it.

hubaig commented Mar 14, 2017

@AirmanAJK and @gclift. Would it be possible for you to write some steps to show how to do this? I get the idea but I am not 100% sure on what file to edit and what to exactly edit in it.

@gclift

This comment has been minimized.

Show comment
Hide comment
@gclift

gclift Mar 14, 2017

@hubaig, sure thing. Here are the steps I performed and it's working like a charm.

STEP 1: Create a new module or use an existing custom module
STEP 2: Create a new Class that extends the Full Action (see below)
STEP 3: Edit the di.xml in the etc directory to include a preference node (see below)
STEP 4: Clear Cache and Reindex

FILE: {company}{module}\Model\Indexer\Product\Category\Action\Full.php

namespace {company}\{module}\Model\Indexer\Product\Category\Action;

class Full extends \Magento\Catalog\Model\Indexer\Category\Product\Action\Full
{
    protected function isRangingNeeded(){
        return false;
    }
}

FILE: {company}{module}\etc\di.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <preference 
            for="Magento\Catalog\Model\Indexer\Category\Product\Action\Full" 
            type="{company}\{module}\Model\Indexer\Product\Category\Action\Full" />
</config>

Note 1: Replace {company} and {module} with your custom module name and directory.
Note 2: This will return all of the products in your store and could cause issues if the number of products is large enough. We have less than 1000 products in our store and had no ill-effects.
Note 3: We are using Magento 2.1.4

Hope this helps.

gclift commented Mar 14, 2017

@hubaig, sure thing. Here are the steps I performed and it's working like a charm.

STEP 1: Create a new module or use an existing custom module
STEP 2: Create a new Class that extends the Full Action (see below)
STEP 3: Edit the di.xml in the etc directory to include a preference node (see below)
STEP 4: Clear Cache and Reindex

FILE: {company}{module}\Model\Indexer\Product\Category\Action\Full.php

namespace {company}\{module}\Model\Indexer\Product\Category\Action;

class Full extends \Magento\Catalog\Model\Indexer\Category\Product\Action\Full
{
    protected function isRangingNeeded(){
        return false;
    }
}

FILE: {company}{module}\etc\di.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <preference 
            for="Magento\Catalog\Model\Indexer\Category\Product\Action\Full" 
            type="{company}\{module}\Model\Indexer\Product\Category\Action\Full" />
</config>

Note 1: Replace {company} and {module} with your custom module name and directory.
Note 2: This will return all of the products in your store and could cause issues if the number of products is large enough. We have less than 1000 products in our store and had no ill-effects.
Note 3: We are using Magento 2.1.4

Hope this helps.

@hubaig

This comment has been minimized.

Show comment
Hide comment
@hubaig

hubaig Mar 14, 2017

@gclift Thank you for taking your time out to write the steps for me. I have a programming foundation and I still am learning about Magento. I am going to see online tutorials on how to create a Magento module and see if I can get this working.

I have about 6000+ items in the online store. Hopefully this won't cause problems.

hubaig commented Mar 14, 2017

@gclift Thank you for taking your time out to write the steps for me. I have a programming foundation and I still am learning about Magento. I am going to see online tutorials on how to create a Magento module and see if I can get this working.

I have about 6000+ items in the online store. Hopefully this won't cause problems.

@AirmanAJK

This comment has been minimized.

Show comment
Hide comment
@AirmanAJK

AirmanAJK Mar 14, 2017

I strongly recommend you set up a proper development environment before making changes. You NEVER want to make programming changes to a production server or modify the core code as a fix. Magento has quite a steep learning curve, especially if you aren't already well-versed in web development. Despite this, it's worth the time to learn how to do it the right way (however painful the experience can be).

AirmanAJK commented Mar 14, 2017

I strongly recommend you set up a proper development environment before making changes. You NEVER want to make programming changes to a production server or modify the core code as a fix. Magento has quite a steep learning curve, especially if you aren't already well-versed in web development. Despite this, it's worth the time to learn how to do it the right way (however painful the experience can be).

@magexo

This comment has been minimized.

Show comment
Hide comment
@magexo

magexo Mar 20, 2017

Contributor

Confirmed. We have same issue on EE 2.1.4

isRangingNeeded workaround helped.

Ranging needs to be fixed.

Contributor

magexo commented Mar 20, 2017

Confirmed. We have same issue on EE 2.1.4

isRangingNeeded workaround helped.

Ranging needs to be fixed.

@hostep

This comment has been minimized.

Show comment
Hide comment
@hostep

hostep Mar 20, 2017

Collaborator

This is most likely a duplicate of #8018

Collaborator

hostep commented Mar 20, 2017

This is most likely a duplicate of #8018

@bangerkuwranger

This comment has been minimized.

Show comment
Hide comment
@bangerkuwranger

bangerkuwranger Apr 13, 2017

For the lazy (or backwards lazy, like myself...)
Composer-installable workaround module, ready for install in CE or EE v2.1.3 and later...
Sorry for the long-ass name

bangerkuwranger commented Apr 13, 2017

For the lazy (or backwards lazy, like myself...)
Composer-installable workaround module, ready for install in CE or EE v2.1.3 and later...
Sorry for the long-ass name

@sylink

This comment has been minimized.

Show comment
Hide comment
@sylink

sylink May 10, 2017

Collaborator

I can confirm overriding to 50k fixed an issue we saw, which isn't ideal but at least proved its a weird bug.

Collaborator

sylink commented May 10, 2017

I can confirm overriding to 50k fixed an issue we saw, which isn't ideal but at least proved its a weird bug.

@andidhouse

This comment has been minimized.

Show comment
Hide comment
@andidhouse

andidhouse Jul 23, 2017

This is still present in 2.1.7 - Major bug for an commerce system but not answered or fixed since month. Wow!

andidhouse commented Jul 23, 2017

This is still present in 2.1.7 - Major bug for an commerce system but not answered or fixed since month. Wow!

@aasim110

This comment has been minimized.

Show comment
Hide comment
@aasim110

aasim110 Aug 2, 2017

@andidhouse try following its working fine for me in 2.1.7

{your magento}/vendor/magento/module-catalog/Model/Indexer/Category/Product/AbstractAction.php
Line number approx : 294

protected function isRangingNeeded()
{
    return false;
}

aasim110 commented Aug 2, 2017

@andidhouse try following its working fine for me in 2.1.7

{your magento}/vendor/magento/module-catalog/Model/Indexer/Category/Product/AbstractAction.php
Line number approx : 294

protected function isRangingNeeded()
{
    return false;
}
@Silarn

This comment has been minimized.

Show comment
Hide comment
@Silarn

Silarn Aug 2, 2017

Official fix exists as MAGETWO-62616 - though it only exists in the devel and 2.2 RC branches.

I'm planning to work on a module version of this fix tomorrow - which is a true fix and not a workaround. (At least until the core fix is released.)

Silarn commented Aug 2, 2017

Official fix exists as MAGETWO-62616 - though it only exists in the devel and 2.2 RC branches.

I'm planning to work on a module version of this fix tomorrow - which is a true fix and not a workaround. (At least until the core fix is released.)

@Silarn

This comment has been minimized.

Show comment
Hide comment
@Silarn

Silarn Aug 2, 2017

I've compiled a standalone version of the official fix at composer package silarn/magento2-catalog-indexer-fix.

For now I just have dev-master available, but as soon as I can confirm this standalone version works I'll push a v1.0.0 tag.

I'm using identical code in a full Magento 2 install, so it should work - I just haven't tested the composer version yet.

Silarn commented Aug 2, 2017

I've compiled a standalone version of the official fix at composer package silarn/magento2-catalog-indexer-fix.

For now I just have dev-master available, but as soon as I can confirm this standalone version works I'll push a v1.0.0 tag.

I'm using identical code in a full Magento 2 install, so it should work - I just haven't tested the composer version yet.

@orlangur

This comment has been minimized.

Show comment
Hide comment
@orlangur

orlangur Sep 20, 2017

Contributor

Closing as the fix for this issue is already available on Magento 2.2.0 branch.

Anyone, do not hesitate to create a backport to 2.1-develop if needed, see https://community.magento.com/t5/Magento-DevBlog/Pull-Requests-for-2-1-x-Patch-Releases/ba-p/65630 for details.

Contributor

orlangur commented Sep 20, 2017

Closing as the fix for this issue is already available on Magento 2.2.0 branch.

Anyone, do not hesitate to create a backport to 2.1-develop if needed, see https://community.magento.com/t5/Magento-DevBlog/Pull-Requests-for-2-1-x-Patch-Releases/ba-p/65630 for details.

@mazestricks

This comment has been minimized.

Show comment
Hide comment
@mazestricks

mazestricks Apr 20, 2018

Thanks it worked on my end! I created a module based on your instructions @gclift Currently using Magento 2.1.3

mazestricks commented Apr 20, 2018

Thanks it worked on my end! I created a module based on your instructions @gclift Currently using Magento 2.1.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment