-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Navigating in the categories tree leads to errors in Redis: "Redis session exceeded concurrent connections" #38851
Comments
Hi @netzkollektiv. Thank you for your report.
Join Magento Community Engineering Slack and ask your questions in #github channel. |
Hi @engcom-Delta. Thank you for working on this issue.
|
Also running into this problem over here. In our case it not only happens when clicking the arrow, but also when just clicking a deeper level of category itself, in that case it's even worse, we sometimes see > 70 similar ajax requests happening simultaneous of which a lot are failing. I think the problem got introduced by 67f1b8f which was released in Magento 2.4.7 and higher, more specifically in this piece of code. I'm not exactly sure what this code is supposed to do, but it sounds wrong to me to perform tens if not hundreds of similar ajax requests for info when you can try to get the necessary info as well within a single request. Maybe @Atul-glo35265 is around to try to explain how that code is supposed to work and if there isn't a way to optimise it so it doesn't send all those ajax requests in a loop, but bundle them up in a single request somehow? |
@hostep Are you sure that this issue is not reproducible on version < 2.4.7? Usually BFF (like graphql) responsibly for batching requests, so implementing batching like |
I can double check with 2.4.6, but since the category tree code was refactored in 2.4.7, I don't think it will trigger on anything lower than 2.4.7. As far as batching goes, I don't think that happens here at the moment. If you indeed would batch like 50 things in a single request, that should significantly improve the situation. @lamskoy: could you open a new issue for your problem? It may sound like the same issue at a glance, but it definitely isn't the same problem (unless those preview calls are generated by the ajax calls discussed in this issue?). |
Hi @netzkollektiv , Verified the issue on 2.4-develop and 2.4.7 and we are not able to reproduce this issue. Preconditions and environment 1.Redis enabled Steps to reproduce:- 1.Go to the backend Redis enabled Admin Manage Categories Open some child-nodes in categories tree logs Please refer the attached screenshots and let us know if we missed anything. |
You need a lot of categories to reproduce the issue, not just 4. See also comment of @hostep |
I have around 2700 categories in tree. This happens |
The issue is not the Redis configuration, the issue is those hundreds (depending on the amount of categories) of concurrent calls to that categoriesJson url, @kandy I hope you understand it better by now? |
Also, seeing this issue. As usual, Magento fails to test at scale. This is probably the 20th time I've seen an issue triggered when there are thousands of records and Magento says "We tested with 2 records and it's fine". Why shouldn't everyone here just move to OroCommerce, BigCommerce, Shopify, Webflow, or one of the other platforms? Magento2 has drastically reduced in quality over the last 5 years. |
Hi @netzkollektiv , Verified the issue on 2.4.7 again. Preconditions 1.Redis enabled Steps to reproduce:- 1.Go to the backend Actual Result: We are getting "Redis session exceeded concurrent connections" error and we can see many XHR requests coming to PageBuilder previews. admin panel "Redis session exceeded concurrent connections" error on logs network logs Could you please let us know for network logs , what kind of error you have reported as it is not mentioned in the issue above. |
Hello, When we want to open the sub-categories there are more than 450 ajax requests to /admin/catalog/category/categoriesJson Why is this issue status is "needs update" ? The issue seems pretty clear to me. Has someone a temporary fix ? |
increase max_concurrency value inside app/etc/env.php. Default value is 6
Bad practice, but could help |
Indeed, that's a workaround for the Redis problem, it won't help with hosting companies that have rate limiters setup though. The refactoring to the category tree that was done in Magento 2.4.7 should be taken a close look at and refactored again so it doesn't require hundreds of ajax calls to get the necessary info. When I first looked at this issue, removing the foreach here also solved the issue, but I have no idea if this has repercussions... |
Hi @netzkollektiv @hostep This particular code is responsible for getting the children of categories at level 3 or further. If we comment out that foreach loop, the triangle button will become invisible and the user won't be able to traverse to categories at level 4. Please refer attached screenshot. We need to check the feasibility of optimizing the code for this issue. Once the issue is confirmed from our internal community team, we will investigate it further. |
Thanks for shedding a bit of light onto the problem! There is no need to wait on confirmation, the issue was already confirmed by many people in this thread. To repeat: you shouldn't try to reproduce the Redis error. You just need to have a big category tree, with many levels, then open the inspector in your browser and see the hundreds of simultaneous ajax requests going to |
Hi @hostep, Thanks for your update. We have verified the issue in Latest 2.4-develop instance and the issue is reproducible.Kindly refer the screenshots. Hence Confirming the issue. Thanks. |
Hello, Internal team has started to work on it Thanks |
Probably fixed by: 1a27a04 ? |
Preconditions and environment
Steps to reproduce
Expected result
No error occurs when navigating through the categories tree in the admin.
Actual result
About half of the requests fail:
=> 500 Internal Server Error
With the following message in exception log:
Additional information
No response
Release note
No response
Triage and priority
The text was updated successfully, but these errors were encountered: