New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import product csv, if categories change, it's adding to the existing instead of replacing #7930

Closed
MattDelac opened this Issue Dec 22, 2016 · 21 comments

Comments

Projects
None yet
@MattDelac

MattDelac commented Dec 22, 2016

Preconditions

  1. Version: Magento CE 2.1.2
  2. Environment: Php7, MariaDB, Varnish, Redis

Steps to reproduce

  1. Create a product with the native import and set the categories as "Root/Category1/Sub_cat1"
  2. Change the categories in the csv as "Root/Category2"
  3. Run a new import with the Add/Update attributes
    selection_274
    selection_275

Expected result

  1. The product now has two categories: "Root/Category1/Sub_cat1" & "Root/Category2"
  2. I would expect to have the product in the new category only instead of an addition.

Actual result

  1. The product should only appear in the category "Root/Category2"

Comments

There is the same issue with the picture, if I run multiple imports with different images, thus the product will have all the images as extra image and the one specified as base image (even if I only have 1 additional image)

Thank you

@nikolaevas

This comment has been minimized.

Show comment
Hide comment
@nikolaevas

nikolaevas Feb 14, 2017

Hi,

Why did you use Add/Update behavior instead of Replace?

How to replace products:

  1. Remove or empty columns store_view_code and _store from csv file. Because products can be replaced only in default scope.
  2. Run a new import with Replace behavior.
  3. Old products will be deleted and new will be added:

Notes:

  • only products with SKUs from your csv file will be replaced
  • new IDs will be generated for replaced products

nikolaevas commented Feb 14, 2017

Hi,

Why did you use Add/Update behavior instead of Replace?

How to replace products:

  1. Remove or empty columns store_view_code and _store from csv file. Because products can be replaced only in default scope.
  2. Run a new import with Replace behavior.
  3. Old products will be deleted and new will be added:

Notes:

  • only products with SKUs from your csv file will be replaced
  • new IDs will be generated for replaced products
@LiamFielding

This comment has been minimized.

Show comment
Hide comment
@LiamFielding

LiamFielding Feb 28, 2017

Because he wanted to update a value as "Add/Update" would appear to be capable of to any reasonable person. If you're saying that the already useless import process that contains no documentation, awful validation is also incapable of this for one specific column...

Have you ever tried importing real products?

LiamFielding commented Feb 28, 2017

Because he wanted to update a value as "Add/Update" would appear to be capable of to any reasonable person. If you're saying that the already useless import process that contains no documentation, awful validation is also incapable of this for one specific column...

Have you ever tried importing real products?

@tsifra

This comment has been minimized.

Show comment
Hide comment
@tsifra

tsifra Apr 19, 2017

Hello, does anyone have solution for this? Very strange behaviour in my opinion. Using "replace" is very strange for such situation. Actualy this means, that i have to use replace function all the time. I guess it will cause problems with products returns and everything that is related with product id.

tsifra commented Apr 19, 2017

Hello, does anyone have solution for this? Very strange behaviour in my opinion. Using "replace" is very strange for such situation. Actualy this means, that i have to use replace function all the time. I guess it will cause problems with products returns and everything that is related with product id.

@Ylmzef

This comment has been minimized.

Show comment
Hide comment
@Ylmzef

Ylmzef Aug 27, 2017

Any solution for this?

Ylmzef commented Aug 27, 2017

Any solution for this?

@MattDelac

This comment has been minimized.

Show comment
Hide comment
@MattDelac

MattDelac Aug 29, 2017

Dear Super Magento team ( @veloraven ),

This bug is from December 2016. Could it be possible to fix it as soon as possible and stop waiting a year for bug updates ?

Thank you in advance
Matt

MattDelac commented Aug 29, 2017

Dear Super Magento team ( @veloraven ),

This bug is from December 2016. Could it be possible to fix it as soon as possible and stop waiting a year for bug updates ?

Thank you in advance
Matt

@magento-engcom-team

This comment has been minimized.

Show comment
Hide comment
@magento-engcom-team

magento-engcom-team Sep 29, 2017

Contributor

@MattDelac, thank you for your report.
We were not able to reproduce this issue by following the steps you provided. If you'd like to update it, please reopen the issue.
We tested the issue on 2.3.0, 2.2.0, 2.1.9

Contributor

magento-engcom-team commented Sep 29, 2017

@MattDelac, thank you for your report.
We were not able to reproduce this issue by following the steps you provided. If you'd like to update it, please reopen the issue.
We tested the issue on 2.3.0, 2.2.0, 2.1.9

@mzenner1

This comment has been minimized.

Show comment
Hide comment
@mzenner1

mzenner1 Nov 15, 2017

This is not fixed in either 2.1.9 or 2.2.x. It appends categories indefinitely. Even if you wipe out the category field in the csv and import, the previously tagged categories will still be there. Using the "replace" method would be a very bad thing as it would be a brand new product and I'm guessing the customer reviews would not follow to the new product.

mzenner1 commented Nov 15, 2017

This is not fixed in either 2.1.9 or 2.2.x. It appends categories indefinitely. Even if you wipe out the category field in the csv and import, the previously tagged categories will still be there. Using the "replace" method would be a very bad thing as it would be a brand new product and I'm guessing the customer reviews would not follow to the new product.

@erfanimani

This comment has been minimized.

Show comment
Hide comment
@erfanimani

erfanimani Nov 23, 2017

This really needs to be reopened.. if I import a product and assign it to category A, and then change my mind, and replace A with B in the import sheet, I expect the product to now be in category B. In reality the product is now in both A and B, which is not what anyone would want.

More specifically, if I first had A in the categories column, but now B, I would like it to update. If I first did not have a categories column at all, but then I imported another sheet with a categories column, I expect the categories data to be added. That's how I interpret the Add/Replace functionality to work.

erfanimani commented Nov 23, 2017

This really needs to be reopened.. if I import a product and assign it to category A, and then change my mind, and replace A with B in the import sheet, I expect the product to now be in category B. In reality the product is now in both A and B, which is not what anyone would want.

More specifically, if I first had A in the categories column, but now B, I would like it to update. If I first did not have a categories column at all, but then I imported another sheet with a categories column, I expect the categories data to be added. That's how I interpret the Add/Replace functionality to work.

@erfanimani

This comment has been minimized.

Show comment
Hide comment
@erfanimani

erfanimani Nov 23, 2017

Hmm, from looking at the code, this seems like this is a feature, not a bug:

        if (Import::BEHAVIOR_APPEND != $this->getBehavior()) {
            $this->_connection->delete(
                $tableName,
                $this->_connection->quoteInto('product_id IN (?)', $delProductId)
            );
        }

Source: https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/CatalogImportExport/Model/Import/Product.php#L1343

I know I'll be patching this on my end.. Suggesting the replace method isn't great since it's very destructive on a live site. By changing the product IDs you'll be blasting away all customer carts, any re-order functionality, product/sales links, reports/statistics, etc.

erfanimani commented Nov 23, 2017

Hmm, from looking at the code, this seems like this is a feature, not a bug:

        if (Import::BEHAVIOR_APPEND != $this->getBehavior()) {
            $this->_connection->delete(
                $tableName,
                $this->_connection->quoteInto('product_id IN (?)', $delProductId)
            );
        }

Source: https://github.com/magento/magento2/blob/2.2-develop/app/code/Magento/CatalogImportExport/Model/Import/Product.php#L1343

I know I'll be patching this on my end.. Suggesting the replace method isn't great since it's very destructive on a live site. By changing the product IDs you'll be blasting away all customer carts, any re-order functionality, product/sales links, reports/statistics, etc.

@sharmstr

This comment has been minimized.

Show comment
Hide comment
@sharmstr

sharmstr Jan 23, 2018

@erfanimani - Did you come up with a patch for this? Thank you.

sharmstr commented Jan 23, 2018

@erfanimani - Did you come up with a patch for this? Thank you.

@erfanimani

This comment has been minimized.

Show comment
Hide comment
@erfanimani

erfanimani Jan 23, 2018

@sharmstr Unfortunately I didn't. I think I ended up deleting the category and reimported everything again.

From what I can remember, fixing the issue wasn't as straightforward as I thought initially...

erfanimani commented Jan 23, 2018

@sharmstr Unfortunately I didn't. I think I ended up deleting the category and reimported everything again.

From what I can remember, fixing the issue wasn't as straightforward as I thought initially...

@sharmstr

This comment has been minimized.

Show comment
Hide comment
@sharmstr

sharmstr Jan 23, 2018

@erfanimani I cant believe it works this way. I'll give fixing it a shot and report back. Thank you.

sharmstr commented Jan 23, 2018

@erfanimani I cant believe it works this way. I'll give fixing it a shot and report back. Thank you.

@priti-rathee

This comment has been minimized.

Show comment
Hide comment
@priti-rathee

priti-rathee Feb 20, 2018

Getting this same error on magento2.2.2(latest version in today's date.)

priti-rathee commented Feb 20, 2018

Getting this same error on magento2.2.2(latest version in today's date.)

@awahab445

This comment has been minimized.

Show comment
Hide comment
@awahab445

awahab445 Apr 12, 2018

@magento-engcom-team , Can you please update on the status of this issue. In which version it got fixed please share. The issue was reported on Dec 2016 and till now, every one getting same issue. Please reopen it and recheck it again.

awahab445 commented Apr 12, 2018

@magento-engcom-team , Can you please update on the status of this issue. In which version it got fixed please share. The issue was reported on Dec 2016 and till now, every one getting same issue. Please reopen it and recheck it again.

@dweerd

This comment has been minimized.

Show comment
Hide comment
@dweerd

dweerd Apr 13, 2018

Can't believe this is closed as 'can't reproduce'. It's very easy to reproduce.

dweerd commented Apr 13, 2018

Can't believe this is closed as 'can't reproduce'. It's very easy to reproduce.

@anebi

This comment has been minimized.

Show comment
Hide comment
@anebi

anebi Apr 20, 2018

I just experienced same issue with Magento 2.2.3 and, yes, it is quite easy to reproduce it. I can't believe that such issue last 2+ more years without fixes.

anebi commented Apr 20, 2018

I just experienced same issue with Magento 2.2.3 and, yes, it is quite easy to reproduce it. I can't believe that such issue last 2+ more years without fixes.

@dweerd

This comment has been minimized.

Show comment
Hide comment
@alen9788

This comment has been minimized.

Show comment
Hide comment
@alen9788

alen9788 Jul 22, 2018

It does still exist for the latest CE 2.2.5 when will it be fixed?

alen9788 commented Jul 22, 2018

It does still exist for the latest CE 2.2.5 when will it be fixed?

@kamilmaliszewski

This comment has been minimized.

Show comment
Hide comment
@kamilmaliszewski

kamilmaliszewski Jul 24, 2018

I have created new issuse with better descirption which passed all tests and reproduced case. You can follow it here.

kamilmaliszewski commented Jul 24, 2018

I have created new issuse with better descirption which passed all tests and reproduced case. You can follow it here.

@mbreckcabinets

This comment has been minimized.

Show comment
Hide comment
@mbreckcabinets

mbreckcabinets Jul 31, 2018

It's very discouraging that Magento can be this obtuse about this issue.

mbreckcabinets commented Jul 31, 2018

It's very discouraging that Magento can be this obtuse about this issue.

@ulftietze

This comment has been minimized.

Show comment
Hide comment
@ulftietze

ulftietze Aug 29, 2018

Contributor

Hey guys,

i stepped into the same problem and solved it with a quite dirty workaorund. May you can use this too.
You have to overwrite the function "processRowCategories" in Magento\CatalogImportExport\Model\Import\Product. All the category links, which are not included in this script are deleted.

Take yourself the time to bind the sku and the categoryIds ^^ Ain't nobody have time for this 💃

/**
     * @param array $rowData
     * @return array
     */
    protected function processRowCategories($rowData)
    {
        $categoriesString = empty($rowData[self::COL_CATEGORY]) ? '' : $rowData[self::COL_CATEGORY];
        $categoryIds = [];
        if (!empty($categoriesString)) {
            $categoryIds = $this->categoryProcessor->upsertCategories(
                $categoriesString,
                $this->getMultipleValueSeparator()
            );

            // **** WORKAORUND START **** \\
            // This is a workaround for really updating the categories and not just simply add product to category
            $categoryNoDeleteIds = implode(",", $categoryIds);
            $deleteString = '
              delete
                ccp
              from
                catalog_category_product ccp
              left join
                catalog_product_entity cpe on ccp.product_id = cpe.entity_id
              where
                cpe.sku = "'. $rowData['sku'] . '"
              and
                ccp.category_id not in (' . $categoryNoDeleteIds .');';
            $this->_connection->query($deleteString);

            // **** WORKAROUND FINISH **** \\

            foreach ($this->categoryProcessor->getFailedCategories() as $error) {
                $this->errorAggregator->addError(
                    AbstractEntity::ERROR_CODE_CATEGORY_NOT_VALID,
                    ProcessingError::ERROR_LEVEL_NOT_CRITICAL,
                    $rowData['rowNum'],
                    self::COL_CATEGORY,
                    __('Category "%1" has not been created.', $error['category'])
                    . ' ' . $error['exception']->getMessage()
                );
            }
        }
        return $categoryIds;
    }
Contributor

ulftietze commented Aug 29, 2018

Hey guys,

i stepped into the same problem and solved it with a quite dirty workaorund. May you can use this too.
You have to overwrite the function "processRowCategories" in Magento\CatalogImportExport\Model\Import\Product. All the category links, which are not included in this script are deleted.

Take yourself the time to bind the sku and the categoryIds ^^ Ain't nobody have time for this 💃

/**
     * @param array $rowData
     * @return array
     */
    protected function processRowCategories($rowData)
    {
        $categoriesString = empty($rowData[self::COL_CATEGORY]) ? '' : $rowData[self::COL_CATEGORY];
        $categoryIds = [];
        if (!empty($categoriesString)) {
            $categoryIds = $this->categoryProcessor->upsertCategories(
                $categoriesString,
                $this->getMultipleValueSeparator()
            );

            // **** WORKAORUND START **** \\
            // This is a workaround for really updating the categories and not just simply add product to category
            $categoryNoDeleteIds = implode(",", $categoryIds);
            $deleteString = '
              delete
                ccp
              from
                catalog_category_product ccp
              left join
                catalog_product_entity cpe on ccp.product_id = cpe.entity_id
              where
                cpe.sku = "'. $rowData['sku'] . '"
              and
                ccp.category_id not in (' . $categoryNoDeleteIds .');';
            $this->_connection->query($deleteString);

            // **** WORKAROUND FINISH **** \\

            foreach ($this->categoryProcessor->getFailedCategories() as $error) {
                $this->errorAggregator->addError(
                    AbstractEntity::ERROR_CODE_CATEGORY_NOT_VALID,
                    ProcessingError::ERROR_LEVEL_NOT_CRITICAL,
                    $rowData['rowNum'],
                    self::COL_CATEGORY,
                    __('Category "%1" has not been created.', $error['category'])
                    . ' ' . $error['exception']->getMessage()
                );
            }
        }
        return $categoryIds;
    }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment