Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.3] Correct flat indexing with staging and > 500 cats #12345

Merged
merged 4 commits into from Dec 1, 2017

Conversation

toddbc
Copy link
Contributor

@toddbc toddbc commented Nov 18, 2017

Description

In a past change, entity_id was used for mapping. However, since the index stores things by entity id (not link id), this prevents it from selecting the correct row.

Additionally, there was a spurious $categoryAttributesData = []; in buildCategoryIndexData() which looks like a mistake, so the argument was ignored.

Please see magento-partners/magento2ce#65 for more detail with regard to Magento Commerce.

Manual testing scenarios

For Magento Open Source:

  1. Start with a fresh DB and create 10 categories.
  2. Create another 500 categories (ideally via script.)
  3. Enable flat category indexing.
  4. Reindex flat category.
  5. Modify the name of certain categories.
  6. Reindex flat category again.

Expected results:
Reindex runs successfully. All rows have correct updates.

Actual results:
Data may not update properly.

Please see magento-partners/magento2ce#65 for the Magento Commerce test scenario.

Contribution checklist

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • All automated tests passed successfully (all builds on Travis CI are green)

These will be replaced with a solution that solves more problems.
Previously, the array was prefilled with entity_id values to ensure those categories
were indexed.  Unfortunately, when using row_id, it was using entity_ids.

When chunking at 500, this could cause an overlapping row to be processed twice (the
second time with missing non-static attributes), and cause a duplicate key error.
This keeps full and rows indexing consistent.
@vkublytskyi vkublytskyi self-assigned this Nov 20, 2017
@vkublytskyi vkublytskyi added this to the November 2017 milestone Nov 27, 2017
@vkublytskyi
Copy link

@toddbc These changes need to be updated as cause an integration test failure which is not part of Travis CI builds (executed by inhouse CI envirotnment):

Magento\Catalog\Model\Indexer\FlatTest::testFlatAfterCreate
Failed asserting that actual size 1 matches expected size 3.

Magento\Catalog\Model\Indexer\FlatTest::testFlatAfterMove
PHPUnit\Framework\Exception: Notice: Undefined offset: 473 in /app/code/Magento/Catalog/Model/Indexer/Category/Flat/Action/Rows.php:178.

Magento\Catalog\Model\Indexer\FlatTest::testMoveCategory
PHPUnit\Framework\Exception: Notice: Undefined offset: 471 in /app/code/Magento/Catalog/Model/Indexer/Category/Flat/Action/Rows.php:178.

@toddbc
Copy link
Contributor Author

toddbc commented Nov 30, 2017

Thanks for catching that - updated.

Is there a way for me to run that same test, or is it supposed to work with EE linked? I must be doing something wrong - magento_integration_tests has EE stuff in it, but didn't reproduce that error through phpunit.

@vkublytskyi
Copy link

@toddbc I reproduced these test failures locally by running single integration test as described at DevDocs (CE+EE installation).

<!-- magento2ce/dev/tests/integration/phpunit.xml -->
<testsuites>
    <testsuite name="PR12345">
        <file>testsuite/Magento/Catalog/Model/Indexer/FlatTest.php</file>
    </testsuite>
</testsuites>

Your last commit fixes an the failure.

@okorshenko okorshenko merged commit d25a9d7 into magento:2.3-develop Dec 1, 2017
okorshenko pushed a commit that referenced this pull request Dec 1, 2017
[EngCom] Public Pull Requests - develop
 - MAGETWO-83289: [Backport 2.3] #11825: Generate new FormKey and replace for oldRequestParams Wishlist #12042
 - MAGETWO-84529: [2.3] Correct flat indexing with staging and > 500 cats #12345
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants