Improve price index consistency and run time

[nicka101]

Description (*)

With a very large product set (~213k) on 1 of our sites, the price index (and through the use of MSI, the inventory index) takes a very long time to complete, with multiple hours spent executing deleteIndexData. It seems the primary cause of this slowdown is string to int conversions or vice versa in MySQL, and ensuring that all IDs are integers beforehand yields much more consistent and reasonable performance on these larger sets

Manual testing scenarios (*)

1. Get a bunch of products
2. Use MSI
3. Have multiple stock sources (with a non default providing most of the stock)
4. Run an inventory index (the issue is most noticeable here due to InventoryCatalog's plugin "PriceIndexUpdatePlugin", and its call to reindexList with a very large number of IDs)

Contribution checklist (*)

• [✓] Pull request has a meaningful description of its purpose
• [✓] All commits are accompanied by meaningful commit messages
• [✓] All new or changed code is covered with unit/integration tests (if applicable)
• [✓] README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• [✓] All automated tests passed successfully (all builds are green)

Resolved issues:

1. resolves [Issue] Improve price index consistency and run time #35719: Improve price index consistency and run time

[m2-assistant]

Hi @nicka101. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

• @magento give me test instance - deploy test instance based on PR changes
• @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE,
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here

ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

For more details, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[nicka101]

@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[nicka101]

@magento run Functional Tests CE, Functional Tests B2B

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[nicka101]

@magento run Functional Tests B2B

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[ishakhsuvarov]

@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Lima]

@magento run Static Tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Lima]

@magento give me 2.4-develop instance

[magento-deployment-service]

Hi @engcom-Lima. Thank you for your request. I'm working on Magento instance for you.

[engcom-Lima]

@magento run Static Tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[magento-deployment-service]

Hi @engcom-Lima, here is your Magento Instance: https://296f7dfe9880199aea9438d6a3548526.instances.magento-community.engineering
Admin access: https://296f7dfe9880199aea9438d6a3548526.instances.magento-community.engineering/admin_f3b0
Login: 9f22087c
Password: 93bf8d259bf1

[engcom-Charlie]

Hi @nicka101,

Thank you for your contribution!

I have fixed static test failures.

Thank you!

[engcom-Charlie]

@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Charlie]

@magento run all tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[engcom-Charlie]

Gone through the build failure. There are some Functional CE, EE, B2B test failures. Below are few test failures.

• AdminCheckingDateAfterChangeFrenchCanadaInterfaceLocaleTest.xml
• AdminSavesTheImagePreviewWithNewNameTest.xml
• AdminAdobeStockStandaloneMediaGallerySavePreviewTest.xml
• AdminStandaloneMediaGallerySearchByKeywordTest.xml

After analysis found that the tests which are failing now are not part of this PR changes or not failing because of any PR change. There are some flecky tests too.

Hence moving the PR for Merging.

Thank you!

[ishakhsuvarov]

@magento run Functional Tests CE, Functional Tests EE, Functional Tests B2B, WebAPI Tests

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.

[ishakhsuvarov]

@magento run Functional Tests EE, Functional Tests B2B

[magento-automated-testing]

The requested builds are added to the queue. You should be able to see them here within a few minutes. Please re-request them if they don't show in a reasonable amount of time.