Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out Of Memory for for attribute filters #53

Merged
merged 1 commit into from May 20, 2019

Conversation

psihius
Copy link
Contributor

@psihius psihius commented Mar 28, 2019

This PR fixes the Out Of Memory error by pushing unique value selection to SQL level instead of iteration in PHP.

The original issue is that the bigger the database was, the more objects were selected from DB, resulting, in our case, in about 300 000 ProductAttributeValue array of objects created in memory and obviously it was not only slow but hit the memory limit fast.
Also excluded attributes where always rendered too, which didn't help cause we had 400k attributes named "Description" that were unique for each product, resulting in absurd memory requirements and hydration taking ages despite the attribute being in the excluded array.

The fix is not perfect - that GROUP BY is going to be slow when there are a lot of values for a given attribute because there are no indexes. And some attributes can be on each product - we for now dealt with it by putting such attributes into the excluded array. Also, boolean/checkbox attributes right now make no sense for filtering and need further development.

Also, doing a GROUP BY on a json, array or text field is, well, stupid, ludicrous, plain and simple - an issue that needs to be handled.

OOM fix during form build for attributes
@bitbager bitbager merged commit 511ebff into BitBagCommerce:master May 20, 2019
@bitbager
Copy link
Member

Thanks, Psihius ;)

@psihius
Copy link
Contributor Author

psihius commented May 21, 2019

You are welcome. Feel free to pick my brain on the further development of this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants