Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Magento 2.2.0rc23: Customer Grid Indexer not working #10838

Closed
jhruehl opened this issue Sep 11, 2017 · 71 comments

Comments

@jhruehl
Copy link
Contributor

commented Sep 11, 2017

The Customer Grid Indexer doesn't work, when reindexing via CLI.

Preconditions

local - Docker Container:

  1. PHP 7.0 and 7.1 (default)
  2. MySQL 5.5.5-10.1.25-MariaDB
  3. Magento 2.2.0rc23

Steps to reproduce

  1. Manage to upgrade your store via composer from 2.1.7 to 2.2.0rc23 (good luck here!)
  2. Run setup:upgrade and fix all problems, which occur during the process until it runs without problems
  3. Run reindex of Cutomer Grid (indexer:reindex customer_grid) via CLI command

Expected result

  1. At this stage only problems occuring, which are caused by plugins from extern modules using deprecated functions, classes, etc.)

Actual result

m220rc23_indexer_customer_grid

@VoidWalker

This comment has been minimized.

Copy link

commented Sep 18, 2017

Same issue on 2.1.8 and 2.1.7

@magento-engcom-team

This comment has been minimized.

Copy link
Contributor

commented Sep 19, 2017

@jhruehl thank you for your report. Unfortunately, we can't guarantee that 3-rd party modules will work with newest Magento versions correctly if they depend on internal Magento code and not on code which marked with @api.
Our team works hard on keeping further Magento versions from incompatible changes, but that's not always possible. If you'd like to update your issue, please reopen it.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 20, 2017

Nonsense. There is no third party module involved. Checked it by deactivating them all. And even the stack trace makes it quite obvious that its not a third party module. And the Customer Grid Indexer works just fine until updating to 2.2.0rc3.0.

So my guess is that you changed the Customer Grid Reindexer process in a way and forgot about Customer Attributes, which have the 'visible in grid' option set. Because that is the most probable reason why the Customer Grid Indexer calls the Attribute Handler.

@orlangur

This comment has been minimized.

Copy link
Contributor

commented Sep 20, 2017

@jhruehl yeah, I was wondering how it is identified as a third-party module issue from the screenshot as well. Is the issue still reproducible for you on latest 2.2.0-preview branch?

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

@orlangur Yes, it is. But I'm going to try to reproduce it now in our vanilla m2.2.0rc3 store test server by adding a customer attribute with 'visible in grid' set to yes.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

Sadly I couldn't reproce the Cutomer Grid Indexer problem with just adding the same attribute to the vanilla store we have already implemented. So I need to investigate further by debugging. But it could be that the massive difference between our productive customer data (where some incoherence snuggled in over the years) and the few (actually two customers), which are set in the vanilla installation.

@VoidWalker

This comment has been minimized.

Copy link

commented Sep 21, 2017

There is a patch for this problem MDVA-4538_EE_2.1.4_v2
Which you can request through partner portal

@orlangur

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

@VoidWalker EE_2.1.4 does not seem to be suitable for 2.2.0rc23. I believe such an old patch (2.1.4 - 2.1.9) is already included in 2.2.0.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

I was actually able to reproduce it in our vanilla m2.2.0rc3.0 store. In my first run, I didn't set 'is_used_in_grid'. Going to open a new Issue, because otherwise it probably gets ignored.

@orlangur

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

@jhruehl could you check the latest 2.2.0-preview please? I'll reopen an issue for you if reproducible.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

@orlangur Isn't the 2.2.0rc3.0 Version the latest Preview?

@orlangur

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

It is the last RC probably, but the branch was updated after that: https://github.com/magento/magento2/tree/2.2.0-preview

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

Just looked at the problematic classes in the 2.2.0 and the 2.2.1-preview. They haven't been updated since then.
'Magento\Customer\Model\Indexer\Source' and '/lib/internal/Magento/Framework/Indexer/Handler/AttributeHandler'

So I don't really see the point to invest a lot of time into an update process to a preview branch, if the function addAttributeToSelect() is still missing in the Source class and still called in the AttributeHandler class.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

So even the folders '/lib/internal/Magento/Framework/Indexer/' and 'Magento/Customer/Model/Indexer/' were not updated for a month, which leads to the conclusion that going into a probably not composer supported update will be just a waste of time leading to the same results.

@orlangur

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

Sure, makes sense to me.

@orlangur orlangur reopened this Sep 21, 2017

@orlangur orlangur removed the non-issue label Sep 21, 2017

@orlangur

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

@magento-engcom-team please provide some insight on #10838 (comment), looks like the issue is reproducible on vanilla installation as well.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

@orlangur Thanks a lot for your help.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

@magento-engcom-team The key to reproduce the problem is setting up a customer eav attribute, which has the 'is_used_in_grid' option set to 1.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 21, 2017

Here is the latest command line stack trace from our vanilla magento 2.2.0rc3.0 installation:
magento2_vanilla@d2372a84f1f8:/var/www/users/magento2_vanilla/src/scripts$ ./magento.sh indexer:reindex customer_grid

Fatal error: Uncaught Error: Call to undefined method Magento\Customer\Model\Indexer\Source::addAttributeToSelect() in /mnt/host/projects/clients/magento2-vanilla/src/www/lib/internal/Magento/Framework/Indexer/Handler/AttributeHandler.php:38
Stack trace:
#0 /mnt/host/projects/clients/magento2-vanilla/src/www/lib/internal/Magento/Framework/Indexer/Action/Base.php(310): Magento\Framework\Indexer\Handler\AttributeHandler->prepareSql(Object(Magento\Customer\Model\Indexer\Source), 'e', Array)
#1 /mnt/host/projects/clients/magento2-vanilla/src/www/lib/internal/Magento/Framework/Indexer/Action/Entity.php(26): Magento\Framework\Indexer\Action\Base->createResultCollection()
#2 /mnt/host/projects/clients/magento2-vanilla/src/www/lib/internal/Magento/Framework/Indexer/Action/Base.php(179): Magento\Framework\Indexer\Action\Entity->prepareDataSource(Array)
#3 /mnt/host/projects/clients/magento2-vanilla/src/www/lib/internal/Magento/Framework/Indexer/Action/Base.php(189): Magento\Framework\Indexer\Action\Base->execute()
#4 /mnt/host/pro in /mnt/host/projects/clients/magento2-vanilla/src/www/lib/internal/Magento/Framework/Indexer/Handler/AttributeHandler.php on line 38

@okorshenko

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

@jhruehl @orlangur thank you for research. we will take a look at this

@okorshenko

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2017

@jhruehl it would be great if you can submit pull request if you know how to fix the issue

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 22, 2017

@okobchenko The solution seems not be just a simple fix, but looking at the code structure a public function addAttributeToSelect() needs to be implemented in the Magento\Customer\Model\Indexer\Source class as there a similar functions like addFieldToSelect() implemented by passing the requests to other injected classes, which have the needed functionality already implemented.

Though it seems that the by dependency injection reachable addAttributeToSelect() functions don't work without further adjustments in that case, because I tried to simply copy them into the problemtatic Source class without real adjustments and this mostly lead to the attribute not being found, because the right tables to join with weren't addressed.

But it seems a good approach would be similar to Magento/Catalog/Model/ResourceModel/Category/Flat/Collection::addAttributeToSelect(), but as I already stated needs to be adjusted to join with the right tables and probably some more.

@VoidWalker

This comment has been minimized.

Copy link

commented Sep 22, 2017

@okorshenko I feel like it make sense to repeat. There is a patch for this issue MDVA-4538_EE_2.1.4_v2
It might be not totally suitable fir 2.2.* but it shows the fix idea

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Sep 28, 2017

Here ist the patch that solves the problem. Though I don't really get why Magento did leave this function out. And if you look at the content of the MDVA-4538_EE_2.1.4_v2 patch, it seems that the 2.1.4 EE version of Magento already used the same indexer logic as CE 2.2 uses now.

It would be really interesting to know what the thought behind that different implementation was.

fix_customer_grid_indexer_add_missing_function 2.txt

@cotardo18

This comment has been minimized.

Copy link

commented Jan 23, 2018

@agata-maksymiuk I agree with you, having an extra attribute on customer's grid is possible only if the attribute type is changed in eav table to static

@cotardo18

This comment has been minimized.

Copy link

commented Jan 23, 2018

@pfortin-expertime answer from @selusi is the good one! if you take my answer you won't be able to save data

@hostep

This comment has been minimized.

Copy link
Contributor

commented Jan 24, 2018

Just FYI: I've got confirmation from the Magento Cloud support that this issue should get fixed in Magento 2.2.4

The fix should be:

diff --git a/vendor/magento/framework/Indexer/Handler/AttributeHandler.php b/vendor/magento/framework/Indexer/Handler/AttributeHandler.php
index 94f34ec..4b74ce5 100644
--- a/vendor/magento/framework/Indexer/Handler/AttributeHandler.php
+++ b/vendor/magento/framework/Indexer/Handler/AttributeHandler.php
@@ -35,7 +35,7 @@ class AttributeHandler implements HandlerInterface
                 'left'
             );
         } else {
-            $source->addAttributeToSelect($fieldInfo['origin'], 'left');
+            $source->addFieldToSelect($fieldInfo['origin'], 'left');
         }
     }
 }

Looks like there is a pending PR which should fix this: #12729

magento-team pushed a commit that referenced this issue Jan 24, 2018
Merge pull request #1987 from magento-panda/PANDA-2.2-develop-PR
Fixed issues:
- MAGETWO-81802: Magento 2.2.0rc23: Customer Grid Indexer not working #10838
- MAGETWO-84448: Product Import does not allow store-specific Custom Option labels
- MAGETWO-85207: Customer's order grid is not displaying
- MAGETWO-86657: Import existing customer with only three columns will override customer group_id and store_id
- MAGETWO-86154: Wrong page cached for logged in user
- MAGETWO-83293: Inefficient SQL query on applying filter on sales order grid
- MAGETWO-83343: Customer shopping cart not accessible in admin order
- MAGETWO-85708: Zoom the image can't be closed by iPhone with Safari browser
VincentMarmiesse added a commit to VincentMarmiesse/magento2 that referenced this issue Jan 31, 2018
Merge pull request magento#1987 from magento-panda/PANDA-2.2-develop-PR
Fixed issues:
- MAGETWO-81802: Magento 2.2.0rc23: Customer Grid Indexer not working magento#10838
- MAGETWO-84448: Product Import does not allow store-specific Custom Option labels
- MAGETWO-85207: Customer's order grid is not displaying
- MAGETWO-86657: Import existing customer with only three columns will override customer group_id and store_id
- MAGETWO-86154: Wrong page cached for logged in user
- MAGETWO-83293: Inefficient SQL query on applying filter on sales order grid
- MAGETWO-83343: Customer shopping cart not accessible in admin order
- MAGETWO-85708: Zoom the image can't be closed by iPhone with Safari browser

MAGETWO-87132: Remove JsTestDriver from Magento codebase

MAGETWO-84219: Creditmemo comment via REST api does not send creditmemo update email

MAGETWO-84219: Creditmemo comment via REST api does not send creditmemo update email

MAGETWO-86452: [Backport for 2.2.x] Update copyright year

alignement array assignement

For Your Eyes Only

Remove whitespace at end of line

FIX: out-of-stock options for configurable product visible on frontend as sellable

Adapt \Magento\Analytics\Block\Adminhtml\System\Config\CollectionTimeLabel::render method to use current locale to render time zone label, as done in \Magento\Framework\Locale\TranslatedLists::getOptionTimezones

Handle transparncy correctly for watermark

The watermark functionality uses imagecopymerge for copying the
watermark into the image, this function loses the alpha information.

Therefor use imagecopy for 100% opacity and use imagefilter in other
cases.

Resolves: magento#10661

Fix handling of watermark with alpha for Imagick

When adding a watermark that already has an alpha channel this was reset
because the overal opacity was set for the image. Instead we should
multiply the original alpha with the requested opacity

Handle non true color images with GD2

Add integration test for watermark with alpha

Added integration tests for handling alpha transparency in watermarks.
Renamed the current test for watermarks as it only tests if the
watermark is correctly places

Fix issues with static tests

Remove ui-state-active from siblings of the expanded menu item

Previously when a menu item was expanded the class `ui-state-active` was
not removed from the previous expanded menu item. This resulted in two
(or more if you expanded more) menu items with this class.

Clean region when select country

🐛 add fallback for Product_links position attribute if position index was not set in API request

👌 shortening variable name

🐛 fix variable re-naming

👌 drop multiple empty lines

👌 closing parenthesis and the opening brace of a multi-line function on the same line

👌 closing parenthesis and the opening brace of a multi-line function on the same line

👌 reset protected variable naming

👌 rename isPositionsSet function to isPositionSet to keep consistency

Fix typo

👌 apply code enhancement

👌 apply code enhancement

Add type hint
@okorshenko

This comment has been minimized.

Copy link
Contributor

commented Feb 27, 2018

The issue has been fixed in 7759827 and delivered in 25e3157

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Feb 28, 2018

Just updated to Magento 2.2.3 CE and it's not fixed. Had to apply my patch to make the customer grid indexer work again.
`scripts/magento.sh indexer:reindex customer_grid

Fatal error: Uncaught Error: Call to undefined method Magento\Customer\Model\Indexer\Source::addAttributeToSelect() in /mnt/host/projects/clients/qualipet/src/www/vendor/magento/framework/Indexer/Handler/AttributeHandler.php:38
Stack trace:
#0 /mnt/host/projects/clients/qualipet/src/www/vendor/magento/framework/Indexer/Action/Base.php(310): Magento\Framework\Indexer\Handler\AttributeHandler->prepareSql(Object(Magento\Customer\Model\Indexer\Source), 'e', Array)
#1 /mnt/host/projects/clients/qualipet/src/www/vendor/magento/framework/Indexer/Action/Entity.php(26): Magento\Framework\Indexer\Action\Base->createResultCollection()
#2 /mnt/host/projects/clients/qualipet/src/www/vendor/magento/framework/Indexer/Action/Base.php(179): Magento\Framework\Indexer\Action\Entity->prepareDataSource(Array)
#3 /mnt/host/projects/clients/qualipet/src/www/vendor/magento/framework/Indexer/Action/Base.php(189): Magento\Framework\Indexer\Action\Base->execute()
#4 /mnt/host/projects/clients/qualipet/src/www/vendor/magento/module-indexer/Model/Ind in /mnt/host/projects/clients/qualipet/src/www/vendor/magento/framework/Indexer/Handler/AttributeHandler.php on line 38`

@hostep

This comment has been minimized.

Copy link
Contributor

commented Feb 28, 2018

@jhruehl: I'm pretty sure the fix is scheduled for 2.2.4 (see my latest comment), 2.2.3 only fixes security issues, no other bug fixes as far as I can see.

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Feb 28, 2018

Here's the patch again for everyone, who still needs a fix, because they have a custom customer attribute, which is used in grid, in their store.

fix_customer_grid_indexer_add_missing_function.txt

@jhruehl

This comment has been minimized.

Copy link
Contributor Author

commented Feb 28, 2018

@hostep Doesn't really matter, if the issue is flagged with 'fixed in 2.2.x'. If they flagged the issue to be fixed, but the fix isn't even given in an update afterwards, the flag is quite misleading, isn't it? Then they should use a 'fixed in 2.2.4' flag and given that, I wouldn't even have reported the error. Again.

@xBitsSec

This comment has been minimized.

Copy link

commented Mar 1, 2018

A Patch in the Core Files is not best idea. You can made a module with this patch.

Yesterday, i made a moudle quickly, to fix this problem. You can find it on composer require vitd/m2_fixes.

Do not forget do following:

  • bin/magento setup:upgrade
  • bin/magento setup:di:compile
  • bin/magento cache:clean
  • bin/magento cache:flush
  • bin/magento setup:static-content:deploy <languages with spaces> -f (on cli will not content deployed in all store used languages, you can use alternative change the deploy:mode:set (developer, production) and back, this work proper).

You can modifying the module on your own wish or you can write an issue ticket on the repo page.

I hope this will help any one :)

@victortodoran

This comment has been minimized.

Copy link

commented Apr 16, 2018

Still broken in 2.2.3 Commerce.

@himanshu8dev

This comment has been minimized.

Copy link

commented Apr 16, 2018

Yes!, I have downloaded and upgrade magento 2.2.2 to 2.2.3. but still issue is not fixed from core.

@t-richards

This comment has been minimized.

Copy link
Contributor

commented May 8, 2018

I can confirm this issue has been fixed in Magento Open Source 2.2.4.

@khasru

This comment has been minimized.

Copy link

commented May 17, 2018

In my case when i create a custom customer attribute and "Add to Column Options" to yes in magento2 EE. i got this error when i try reindex.
So that i set this field value to no. and its working fine.

Magento Team needs to taking care it.

@hafizjee

This comment has been minimized.

Copy link

commented Sep 18, 2018

i have add this function public function addAttributeToSelect($attribute, $joinType = false) { $this->customerCollection->addAttributeToSelect($attribute, $joinType); return $this; } in app/code/Magento/Customer/Model/Indexer/Source.php
but got an error while re indexing.
PHP Fatal error: Uncaught Error: Call to undefined method Magento\Customer\Model\Indexer\Source::addAttributeToSelect() in /var/www/html/ncsmokehouse/vendor/magento/framework/Indexer/Handler/AttributeHandler.php:38
what m i do ?

@selusi

This comment has been minimized.

Copy link

commented Sep 18, 2018

i have add this function public function addAttributeToSelect($attribute, $joinType = false) { $this->customerCollection->addAttributeToSelect($attribute, $joinType); return $this; } in app/code/Magento/Customer/Model/Indexer/Source.php
but got an error while re indexing.
PHP Fatal error: Uncaught Error: Call to undefined method Magento\Customer\Model\Indexer\Source::addAttributeToSelect() in /var/www/html/ncsmokehouse/vendor/magento/framework/Indexer/Handler/AttributeHandler.php:38
what m i do ?

Before updating the index you have to launch the following commands:

php bin/magento setup:upgrade
php bin/magento setup:di:compile
@hafizjee

This comment has been minimized.

Copy link

commented Sep 18, 2018

i have add this function public function addAttributeToSelect($attribute, $joinType = false) { $this->customerCollection->addAttributeToSelect($attribute, $joinType); return $this; } in app/code/Magento/Customer/Model/Indexer/Source.php
but got an error while re indexing.
PHP Fatal error: Uncaught Error: Call to undefined method Magento\Customer\Model\Indexer\Source::addAttributeToSelect() in /var/www/html/ncsmokehouse/vendor/magento/framework/Indexer/Handler/AttributeHandler.php:38
what m i do ?

Before updating the index you have to launch the following commands:

php bin/magento setup:upgrade
php bin/magento setup:di:compile

i already launch these commands. but no success

@hafizjee

This comment has been minimized.

Copy link

commented Sep 18, 2018

I apologize because in the previous comment I indicated that the
Magento \ Customer \ Model \ Indexer \ Source.php file did not exist, while it is actually alive and well.

However, by modifying the indexer.xml file the index rebuilding work fine.

Alternativly is possible add the below function to the file Source.php:

public function addAttributeToSelect($attribute, $joinType = false)
{
    $this->customerCollection->addAttributeToSelect($attribute, $joinType);
    return $this;
}

I tested both methods on my test site.

m follow this but got an error . how can i resolve ?

@hostep

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2018

@hafizjee, the correct fix is this one: 7759827

If you use composer, you can follow this guide to install this as a patch on top of your installation: https://support.magento.com/hc/en-us/articles/360005484154-Create-a-patch-for-a-Magento-2-Composer-installation-from-a-GitHub-commit

Or you can just upgrade to Magento 2.2.4 or higher where this fix is included.

@selusi

This comment has been minimized.

Copy link

commented Sep 18, 2018

@hafizjee
You can try to delete the folders:
generated\code
var\cache
and then launch:

php bin/magento setup:upgrade
php bin/magento setup:di:compile

The error code leaves no doubt of interpretation. The application does not find the required method, so either you copied the code to the wrong file or you did not completely delete the cache.
Another possible cause could be the permissions assigned to the folders that could prevent the proper rewriting of the cache.

@matinict

This comment has been minimized.

Copy link

commented Sep 30, 2018

Here's the patch again for everyone, who still needs a fix, because they have a custom customer attribute, which is used in grid, in their store.

fix_customer_grid_indexer_add_missing_function.txt

work for me

@gjportegies

This comment has been minimized.

Copy link

commented Nov 14, 2018

I'm experiencing this problem on my Magento 2.1.12 as well. Is there any fix for the Magento 2.1.* ? Because the fixes mentioned earlier are all applying for Magento 2.2+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.