Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BOOM-4298] Large number of product combinations block browser #9809

Open
prestonBot opened this issue Aug 22, 2018 · 77 comments
Open

[BOOM-4298] Large number of product combinations block browser #9809

prestonBot opened this issue Aug 22, 2018 · 77 comments
Labels
1.7.2.3 1.7.4.0 1.7.4.1 BO Bug Combinations Major Must-have Performance Products Ready Topwatchers

Comments

@prestonBot
Copy link
Collaborator

prestonBot commented Aug 22, 2018

This issue has been migrated from the Forge. Read the original ticket here.

  • Reporter: laurentsupport
  • Created at: Tue, 14 Nov 2017 11:35:21 +0100

Hello,

we have a client site conntected with icecat database.
At now it haves 4 active shops and abut 60000 articles. From icecat we have about 7500 features.
When client connect to backoffice and operate on product the browser blocks because the Features tab are too large.
Is there a best way to paginate and manage features in admin product page?

Kind regards,

  • How to reproduce the issue ?

N/A

@prestonBot prestonBot added 1.7.2.3 1.7.4.0 1.7.4.1 Bug In Progress Major Performance labels Aug 22, 2018
@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: salma.moakhar
  • Created at: Tue, 14 Nov 2017 14:42:34 +0100

Hi Laurent Ndaw,

We tried to work on the pagination declensions but it's not over yet.
We will also work to lighten the product page, it will take time, so unfortunately we have no solution currently.
I close the ticket, feel free to open a new ticket when needed.

Thanks & best regards, Salma

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: lila75
  • Created at: Fri, 26 Jan 2018 17:46:02 +0100

I have the same problem, and you can't do anything for us ?

So i have to close my prestashop account and go to see magenta or another one ?

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: khouloud.belguith
  • Created at: Fri, 26 Jan 2018 17:55:31 +0100

Hi,

We are aware about this issue and we are working on it.
No, this issue is in progress, it is assigned to our developer Jocelyn Fournier, he is working on it.
Thanks for your patience

Best regards, Khouloud

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: lila75
  • Created at: Fri, 26 Jan 2018 18:02:02 +0100

is it possible to contact her ?

i'm in late with my work...

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: khouloud.belguith
  • Created at: Fri, 26 Jan 2018 18:07:09 +0100

Hi ,

You can find his address mail Jocelyn Fournier in his profile

Best regards, Khouloud

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: lila75
  • Created at: Fri, 26 Jan 2018 18:11:10 +0100

I don't know where is her profile.

Can you send me her address mail please?

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: lila75
  • Created at: Fri, 26 Jan 2018 18:12:58 +0100

sorry, i find it.

 

Thank your for your reactivity...

best regards,

Lila

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: yoan - consult'in web
  • Created at: Tue, 27 Feb 2018 10:46:00 +0100

Hello,

 

I have the same problem withmy customer, is it possible tu have a return about this resolving?

 

Thanks Yoan

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: joce
  • Created at: Tue, 27 Feb 2018 14:02:43 +0100

boom-2950 needs to be fixed first

@prestonBot
Copy link
Collaborator Author

prestonBot commented Aug 22, 2018

This comment has been migrated from the Forge. Read the original comment here.

  • Author: root
  • Created at: Wed, 18 Apr 2018 07:36:18 +0200

boom-2950 needs to be fixed first

Ok mr.Fournier,
now you fixed attribute saving problem in BOOM-2950
It is time to fix this one 4298 and take a look on similar behavior I reported at http://forge.prestashop.com/browse/BOOM-5350

I remember similar error on 1.6.* with endless spinning save button
without possibility to have scaleable [server side strength related] number of categories and products
Prestashop will lose it's competitiveness with other platforms.

@mpittiloni
Copy link

mpittiloni commented Nov 8, 2018

Hello, do you have any update on this issue? It's starting to be a very important...

@mchbani
Copy link

mchbani commented Nov 12, 2018

Hi
I have the same issue with 1.7.4.4. any update please?

@khouloudbelguith
Copy link
Contributor

khouloudbelguith commented Nov 12, 2018

Hi all,

This issue is in progress, it is assigned to our developer.
Thanks for your patience & your understanding!

@tengav
Copy link

tengav commented Nov 23, 2018

When we fixed that, the problem seems to be longer.
Is there a workarround here?

@JanicW
Copy link

JanicW commented Jan 31, 2019

Hello, is there any news about this bug. I really need to add more than 390 combinations.

@blaz-blazer
Copy link

blaz-blazer commented Feb 26, 2020

Thanks for info @marionf. Do you have an ETA for version 1.7.8? I know it's hard to predict but just an estimate.

Blaz

@marionf
Copy link
Contributor

marionf commented Feb 26, 2020

The 1.7.8 is planned for the end of the year

@maofree
Copy link

maofree commented Feb 29, 2020

hi
this is a big problem, at this moment that page cannot manage many data
1
3

I've seen if there are many combinations and many images I need to associate only a part of images for each combination, to get the possibility to save the changes.
I thought I had solved but today I get the same error, to solve I need to remove all combinations and start again and reduce the data for each combination, It is not possible to wait other 10 months to solve this problem

@maofree
Copy link

maofree commented Feb 29, 2020

that error happens with 1 language, to solve it I need to remove all combinations are restart again setting less images (or removing other things).
removed all combinations the page returns to work.
Now I've added a new language, so the error returns also without combinations.
it is not possible to work in this mode for site with many data for some products

after some tests I solve this new error, removing all text from the long descriptions of both languages

so the problem is the limits on this page

@ahmelg
Copy link

ahmelg commented Jul 8, 2020

I have the same problem. I have version 1.7.6.5 with products with 702 combinations, it crashes when I save.
Have you solved the problem?

@5koder
Copy link

5koder commented Jul 20, 2020

@ahmelg same here. I really need a workaround

@jbenezech
Copy link
Contributor

jbenezech commented Sep 10, 2020

A dirty workaround for those who can afford it: jbenezech@004cc66

Of course you won't be able to update combinations from the product page and of course you'll have to re-apply the patch when upgrading prestashop, but until they finalize their update (with pagination), you should at least be able to edit products normally.

@salimmejdoub
Copy link

salimmejdoub commented Jan 11, 2021

Hello,
When you have a huge number of categories and sub-categories,
Parent category get sub-categories and each sub-category get parent category ==> infinite loop
src/Adapter/CategoryCategoryDataProvider.php from line

Prestashop version : 1.7.7
Name of function : getParentNamesFromList($categoryId)
Line : 204
Code :

    $parent_catagories_array = [];
    while (isset($this->categoryList[$categoryId])) {
        $category = $this->categoryList[$categoryId];
        $parent_category_id = $category['id_parent'];
        /* exit from infinite loop */
        if (in_array($parent_category_id, $parent_catagories_array)){ break;} 
        $parent_catagories_array[] = $parent_category_id;
        $categories[] = $category['name'];
        $categoryId = $parent_category_id;

        $categoryId = $category['id_parent'];
    }

@marek1545
Copy link

marek1545 commented Feb 4, 2021

Hello,
I observe this issue on 2 years from when I migrate my shop from PS 1.6 to 1.7. Nothing change, but I still waiting with hope.

New method of create combinations in PS 1.7 in my opinion is little not good enough. First you create combination and next you edit price e.t.c. On start you have to spent many time to fill information for every combination. I understand for better customization generator from 1.7 is better. In PS 1.6 was faster to create combination and sale product.

Maybe the best Idea will be merge solve from PS 1.6 and 1.7.
To generate easy combinations on start with price and weight. Add option not only sum attributes, but second option get max price from few selected attributes.

I modify default PS generator to do this. Now I can generate thousandth combinations for my product in few minutes in Prestashop 1.7. I create that modify 2 years ago and from that time I thing is was good idea.
Only edit is little problem. However I limit init combination and edit :) When I must change price I regenerate combinations. Is dirty workaround, but working. I can sell all my products, not only part.

I was have plan to do pagination on my own, but I believe in you. If you add pagination to next upgrade you solve problems many user. Maybe in second upgrade you listen my advice and add this merged idea to functionality.

Screenshot_2

Sorry for my English.

@matks
Copy link
Contributor

matks commented Apr 24, 2021

Ping @jolelievre @MatShir this is EXACTLY what we aim to fix with Experimental Product Page !

@lim3ra
Copy link

lim3ra commented Nov 18, 2021

Do we have any progress on this? I am having 500 errors on product edit pages and it seems it is related. Presta 1.7.7.8

@maofree
Copy link

maofree commented Nov 18, 2021

Hi
try to upgrade your site to 1.7.8.1. do it on a clone website to be sure all modules are compatible

@jolelievre
Copy link
Contributor

jolelievre commented Nov 18, 2021

The product page is currently under re-development It will be available in the next version (8.0) however it is partially available in 1.7.8 as an experimental feature. The page was not fully finished but it is accessible by enabling the feature in Advanced Parameters > Experimental Features you can then enable the experimental product page.

From here you can then access it via additional buttons, when you are on your product page a button to switch to the new page Edit on experimental page this will give you a preview of the coming new product page

You can also try and test it on the develop branch where the page has even more features.

@maofree
Copy link

maofree commented Nov 18, 2021

Hi @jolelievre
can you show me the extra buttons when I set the new experimental option?
I cannot see them in the product page

@jolelievre
Copy link
Contributor

jolelievre commented Nov 18, 2021

Hi @maofree and @lim3ra

Here is a short video that shows how to:

  • enable the experimental feature
  • access the experimental page from the "legacy page"
  • quick display of the combinations generation and filtering in the experimental page

https://watch.screencastify.com/v/XkmPfNZymH8ToENFQkJf

So the new page is more optimized for products with a big number of combinations. However, we didn't test this with thousands of feature values (FeatureValue and Attribute are not the same thing). It seems like this issue is talking about both topics so I'm not sure it answers your need exactly.

In combination we now have pagination and some filtering features so it should improve performance a lot (or at the very least avoid blocking the page).

For features we didn't change much, meaning the list of all Features is loaded for each page (but the FeatureValue are loaded asynchronously after selection of the feature). So if you have 35000 different feature values they won't be all loaded at once (only feature by feature). However, if you have 15000 different features they will all be loaded on page load.

It might need to be improved (by searching the Feature via an API instead of loading the whole list for example), but we need feedback from you to know if it is necessary or not so don't hesitate to test this experimental page and provide us feedback. If we have time we may be able to fix/improve it before the 8.0 is released, but without feedback, we can't know what is performant enough or not.

@maofree
Copy link

maofree commented Nov 19, 2021

thanks @jolelievre
understood how to find it

I've found a bug on the new combination tab pushing on the pencil icon

#26675

bye

@lim3ra
Copy link

lim3ra commented Nov 27, 2021

Prestashop 1.7.8.1
PHP 7.4.24

Using new, experimental product page, still I can see this issue:

`Error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes)

in FeatureRepository.php line 100`

do we have any other ideas?

@jolelievre
Copy link
Contributor

jolelievre commented Nov 29, 2021

@lim3ra can you tell me how many features and feature values you have in your database please?

@lim3ra
Copy link

lim3ra commented Nov 29, 2021

ps_feature_value and ps_feature_product have around 130k records each

@jolelievre
Copy link
Contributor

jolelievre commented Nov 29, 2021

Ok, so there is no magical solution for such a large list 😅

Currently, we don't display all the feature values, however we create a select box with the whole list of Features. What we could do is not prefill this select bow and instead perform a search while the user is typing, but the drawback of this solution is that you can no longer scroll the list of features...

Then once the feature is selected we populate the feature values selector, again here if you have too many values there is a possibility of this update failing as well for lack of memory. Then we could make the value selector a search input as well but here we get the same drawback that you can't scroll the select bow anymore.

It's a tough one, because although I know this kind of feature would be case for shops which have 10k or 100k values, but it might make the feature less usable for small shops as well 🤔

@lim3ra
Copy link

lim3ra commented Nov 29, 2021

I think I found the issue

SELECT * FROM ps_feature_productWHEREid_feature = 0; returns 130k records
SELECT * FROM ps_feature_valueWHEREid_feature = 0; - same

just to compare

SELECT * FROM ps_feature_valueWHEREid_feature != 0; returns ~2k records

IDK if there is some kind of bug but I have a feature with ID 0 duplicated thousands of times. I am going to clean DB, remove all entries about the feature with ID 0 and we will see what will happen.


update: yup, it is fixed now. Thank you all for help, seems my problem was not really related to this topic after all 🙃

@jolelievre
Copy link
Contributor

jolelievre commented Nov 29, 2021

Ohh this is very weird indeed, id_feature == 0 should not be possible since it's supposed to be an unsigned primary key with auto-increment Maybe your Mysql database is not correct, maybe you should check its structure but this is how the feature table is created:

CREATE TABLE `PREFIX_feature` (
  `id_feature` int(10) unsigned NOT NULL auto_increment,
  `position` int(10) unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`id_feature`)
) ENGINE=ENGINE_TYPE DEFAULT CHARSET=utf8mb4 COLLATION;

So id_feature should never be 0, let's say that for some reason one row was created with this ID it should be unique because of the primary key.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.7.2.3 1.7.4.0 1.7.4.1 BO Bug Combinations Major Must-have Performance Products Ready Topwatchers
Projects
None yet
Development

No branches or pull requests