WebAPI products are not removed from Elasticsearch

[magento-engcom-team]

In WebAPI testing, I saw that products were being added to elasticsearch but never removed. I'm unsure if this is normal. Many products are being deleted but some are not. This causing a slow increase in Elasticsearch memory usage. I specifically looked at GraphQL.

<2020-07-06T18:09:13,718][INFO ][o.e.c.m.MetaDataCreateIndexService] [139ed5ecd08c] [magento2*product_8*v2] creating index, cause [api], templates [], shards [1]/[1], mappings []

[2020-07-06T18:09:13,809][INFO ][o.e.c.m.MetaDataMappingService] [139ed5ecd08c] [magento2*product_8_v2/IwEuwlSSQCCfAf7K5Qq-RA] create*mapping [document]

[2020-07-06T18:09:14,220][INFO ][o.e.c.m.MetaDataDeleteIndexService] [139ed5ecd08c] [magento2*product_8*v2/IwEuwlSSQCCfAf7K5Qq-RA] deleting index

[2020-07-06T18:09:14,270][INFO ][o.e.c.m.MetaDataCreateIndexService] [139ed5ecd08c] [magento2*product_8*v2] creating index, cause [api], templates [], shards [1]/[1], mappings []

[2020-07-06T18:09:14,366][INFO ][o.e.c.m.MetaDataMappingService] [139ed5ecd08c] [magento2*product_8_v2/AJNQ5MhrQKev8HiQWS3GDA] create*mapping [document]

[2020-07-06T18:09:15,596][INFO ][o.e.c.m.MetaDataMappingService] [139ed5ecd08c] [magento2*product_8_v2/AJNQ5MhrQKev8HiQWS3GDA] update*mapping [document]

[2020-07-06T18:09:16,649][INFO ][o.e.c.m.MetaDataMappingService] [139ed5ecd08c] [magento2*product_8_v2/AJNQ5MhrQKev8HiQWS3GDA] update*mapping [document>

yellow open magento2*product_8*v2 AJNQ5MhrQKev8HiQWS3GDA 1 1 0 0 71.8kb 71.8kb

[magento-engcom-team]

The issue was exported from the internal JIRA. The link to the original JIRA issue: https://jira.corp.magento.com/browse/MC-35790

[m2-assistant]

Hi @magento-engcom-team. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[magento-engcom-team]

✅ Confirmed by @magento-engcom-team
Thank you for verifying the issue. Based on the provided information internal tickets MC-35790 were created

Issue Available: @magento-engcom-team, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[github-jira-sync-bot]

❌ Cannot export the issue. This GitHub issue is already linked to Jira issue(s): https://jira.corp.magento.com/browse/AC-742

[hostep]

We've seen a similar problem on one of our shops, where sometimes products are not deleted from the Elasticsearch index after a product import where products get deleted on Magento 2.4.3-p1.
We do use https://github.com/Smile-SA/elasticsuite/ on this particular project, so it might also be a bug in that module, but it's hard to say at this moment ...

@sdzhepa or @sidolov or @sivaschenko: is there any news yet around this ticket (MC-35790 or AC-742)? Or do investigations still need to start?