Duplicate Attribute Option Value

[webdlp]

I had duplicate attribute option value issue for configurable products. My website is running magento 1.901 and magmi 0.7.19a. Everything seems running fine except duplicate attribute option values for configurable products. For example, I imported a configurable product with 5 colors (Blue, Gold, Green, Pink, Red) with configurable_attributes as color. The product was imported successfully. But it also created three new but duplicate attribute option values for color (Blue, Gold, Pink), which I had set up before. The other two color option values (Green and Red) still used the original values without duplication, which are correct. That is very weird.

I did not find Capital and white space issues. I also turned on value trimmer, which did not help. Therefore, I'd like to hear some advices from our magmi experts

The following are plugins I used to import configurable products.

CSV Datasource (1.3.1) by Dweeves
On the fly category creator/importer (0.2.3) by Dweeves
Image attributes processor (1.0.25) by Dweeves
On the fly indexer (0.1.5) by Dweeves
Tier price importer (0.0.9a) by Dweeves,bepixeld
Generic mapper (0.0.6a) by Dweeves
Configurable Item processor (1.3.7a) by Dweeves
Standard Attribute Import (1.0.6) by Dweeves

Thanks!

[light4ever]

Webdlp,

I'm not aware of a Standard Attribute Import plugin. Where did you find that plugin or is that plugin the "Mass Select/Multiselect Attribute value translater" utility plugin?

I had a very similar problem in magneto 1.8, in my case when I looked at the attribute on the admin page I get a exception.

[webdlp]

In magmi 0.7.19a, all these utilities are included.

[webdlp]

I found duplication is not only just for attributes, but also for product tags.

[kbouh320]

Hello,

We encounter the same issue here (v0.7.19a and magento CE 1.9). Attribute values are duplicated during import.

[Centilitre]

Hello all and @dweeves

I also have a similar issue today and I have done some testing about this. Here is what I have found:

1. It happens when I use super_attribute_pricing (see csv screenshot), if I remove this field, it is fine.
   
2. There are two problems when I use super_attribute_pricing:
   • incorrect option prices:
     
   • Duplicate Attribute Option Value, similar to what OP says, but the duplicated value is always 'black' in my case, even if I change the color value in the csv file.
     
3. I have tested the csv file on v0.7.18, my own master branch (v0.7.20 I think) and the latest brand new v0.7.21_git (on magento 1.7.0.2 and 1.9.0.1). The issue doesn't happen on v0.7.18. So I guess a bug was made in v0.7.19a, and very likely in super_attribute_pricing handling code of configurable processor.

[dweeves]

fixed by latest git commit, also you can use advanced utility, custom SQL, "remove orphan option values" option to fix the existing db.

[Centilitre]

Thanks for the fix.

[kbouh320]

Thank you dweeves !

[webdlp]

I had tried v0.7.19a, v0.7.20 and v0.7.21_git on my magento v 1.901. The duplicate issues are still there. Which version do you guys use to fix the duplicate issue? Thanks!

[kbouh320]

Hello,

The same here, I tested this afternoon the v0.7.21_git on a 1.901 magento, and the problem is still there for me.

Kevin

[dweeves]

I think new duplicates won't be created but you need first to remove existing ones & perform à new import.

[webdlp]

Hi, dweeves,
That is I did. I delete all duplicates and perform a complete new import with all new skus. The duplicate atrribute option value will appear after magmi import.

[kbouh320]

Hello all,

I tested with a fresh catalog and no attribute values, I have the same issue.

Kevin

[dweeves]

Ok, so i'll need to strengthen my unit test. could you post some basic sample files that cause your problem, it may help.
Are you using : multistore translation syntax ? is it happening on "configurable_attribute" or standard select attribute? is it happening on "multiselect" attribute ?

I need a much information as i can on the setups on which this bug is happening.

[webdlp]

I just have a configurable product with color as attribute. Here is the sample csv file I used to import. The products can be listed successfully but also create duplicate color options such as blue, purple, multicolor and pink:

websites,store,attribute_set,type,sku,categories,name,description,visibility,qty,image,small_image,thumbnail,color,configurable_attributes
base,admin,Fashion Jewelry Attribute Set,simple,HCC195-Blue,Fashion Jewelry/Hair Accessories/Hair Pins,Test HCC195 Blue,"

Product #: HCC195
Color options: blue
Quantity: 1
Condition: new

",Not Visible Individually,5,+/HC195-01-600.jpg,/HC195-01-600.jpg,/HC195-01-600.jpg,Blue,
base,admin,Fashion Jewelry Attribute Set,simple,HCC195-Purple,Fashion Jewelry/Hair Accessories/Hair Pins,Test HCC195 Purple,"

Product #: HCC195
Color options: purple
Quantity: 1
Condition: new

",Not Visible Individually,5,+/HC195-02-600.jpg,/HC195-02-600.jpg,/HC195-02-600.jpg,Purple,
base,admin,Fashion Jewelry Attribute Set,simple,HCC195-Multicolor,Fashion Jewelry/Hair Accessories/Hair Pins,Test HCC195 Multicolor,"

Product #: HCC195
Color options: multicolor
Quantity: 1
Condition: new

",Not Visible Individually,5,+/HC195-03-600.jpg,/HC195-03-600.jpg,/HC195-03-600.jpg,Multicolor,
base,admin,Fashion Jewelry Attribute Set,simple,HCC195-Pink,Fashion Jewelry/Hair Accessories/Hair Pins,Test HCC195 Pink,"

Product #: HCC195
Color options: pink
Quantity: 1
Condition: new

",Not Visible Individually,5,+/HC195-04-600.jpg,/HC195-04-600.jpg,/HC195-04-600.jpg,Pink,
base,admin,Fashion Jewelry Attribute Set,configurable,HCC195,Fashion Jewelry/Hair Accessories/Hair Pins,Test HCC195,"

Product #: HCC195
Color options: blue, purple, multicolor, pink
Quantity: 1
Condition: new

","Catalog, Search",,+/HC195-01A-600.jpg,/HC195-01A-600.jpg,/HC195-01A-600.jpg,,color

[dweeves]

I made some improvements on latest git commit. could you first use the "Advanced utilities/Custom Magento SQL Utility v1.0.3" and select the "Remove specific orphan option values" option of this utilty to cleanup the "color" attribute ?
This should remove "unused" duplicates and end with a clean list.
Then perform an import with the latest git. I made some extra unit test for "configurable attributes" used as options and i had no such behaviour.

[webdlp]

some html tags do not show in Description in above file.

[webdlp]

dweeves, I will try as you suggest.

[webdlp]

I ran "Advanced utilities/Custom Magento SQL Utility v1.0.3" and got this test result:
MAGMI by dweeves - version:0.7.21_git

Running Magmi Utilities Engine v1.0.1 by dweeves

Magento Mass Importer by dweeves - version:0.7.21_git

Custom Magento SQL Utility-1.0.3-Dweeves-

But this script did not remove the duplicate options. Apparently, those duplicate options have at least one product associated with them - the one I just imported. To remove those duplicate options, I have to manually delete the options and save configuration for attribute "color".

I updated magmi-git (the lastest one, three days old) as suggested. The duplicate options are still showing up after import. Do you think that my csv file may have some problem?

[dweeves]

maybe the attribute "color" itself may have a problem. it should be set
"global" scope & "use for configurable" to be used as configurable
attribute through magmi.

could you send the "raw" csv file to dweeves@gmail.com ?

2014-11-24 15:48 GMT+01:00 webdlp notifications@github.com:

I ran "Advanced utilities/Custom Magento SQL Utility v1.0.3" and got this
test result:
MAGMI by dweeves - version:0.7.21_git

Running Magmi Utilities Engine v1.0.1 by dweeves

Magento Mass Importer by dweeves - version:0.7.21_git

Custom Magento SQL Utility-1.0.3-Dweeves-

But this script did not remove the duplicate options. Apparently, those
duplicate options have at least one product associated with them - the one
I just imported. To remove those duplicate options, I have to manually
delete the options and save configuration for attribute "color".

I updated magmi-git (the lastest one, three days old) as suggested. The
duplicate options are still showing up after import. Do you think that my
csv file may have some problem?

—
Reply to this email directly or view it on GitHub
#126 (comment).

[webdlp]

The setting was correct for attribute "color". I used this attribute to create many configurable products in the backend with no issue. I sent a sample csv file to your e-mail. Please check. Thank you!

[dweeves]

I totally refactored the option management. i cannot produce a case that generate duplicates with my unit tests. could you check latest ?

[damirock]

Dear @dweeves

Thank you for this amazing development. Will definitely make a donation to your project, saved so much time for our project.
There's a single problem i'm experiencing right now - it's duplicate attribute values which appeared somehow during last import processes. Unfortunately, i can't tell now which import caused that, but i'm sure i didn't change my input data format, it's always the same.
I'm attaching CSV file i used to import to this thread, please let me know what you think.
https://www.dropbox.com/s/ecpirxcasbc0jsy/magento_import_22_example.csv?dl=0

Screens:

Best regards,
Damir

[damirock]

Dear @dweeves, any news here? Could you replay this behavior?

[barryvdh]

I have a similar issue with the latest git version:

Item (Mage_Eav_Model_Entity_Attribute_Option) with the same id "1234" already exist

Is this related? For example, admin = store_id 0, nl=1, en=2, de=3. After import, I should have 0, 2 and 3 as options, but for store 2 and 3, a duplicate option is added.. This will fail on the frontend and stop the search index..

[webdlp]

It has been almost half year past since I first had this problem.

Recently, I started to change my template to the rwd template and had problem in color swatch for certain colors. These colors were exactly the same with the duplicate colors I had encountered several months ago with Magmi. Since the color swatch is not related to Magmi, I believed there was some difference between the colors worked and the colors did not work. Finally, I found that those did not work had a white space in my database, which was created during attribute setup. That explains why I had value trimmer but still had the duplicate issue.
If you encounter the same problem, please check your database or backend to see whether your attribute option value has the white space, not just check the white space in CSV. That is the lesson I learned.
Also, I'd like to thank dweeves for developing such a nice tool which make import such a easy job.

[satyaranjan1231984]

I am new in Magmi. I am facing a issue while going to import the configurable products. Duplicate attribute option value are getting inserted. I am using Magmi v0.7.21.
Please help me to resolved it.

[ghost]

I have a similar issue with the latest git version:

[dweeves]

It seems the values are really different , so no real duplicates.

"Alcaltel Lucent" <> "Alcatel-Lucent" ( - )
"Arista Networks" <> "AristaNetworks" (space)
"Aruba Networks" <> "ArubaNetworks" (space)
'Black Box" <> "BlackBox" (space)

2016-03-21 7:49 GMT+01:00 Chetan Bura notifications@github.com:

I have a similar issue with the latest git version:

[image: attribute]
https://cloud.githubusercontent.com/assets/8872016/13912086/27bf18de-ef5f-11e5-86e5-064cf5f427a7.png

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#126 (comment)

[apothegm]

I'm having a similar issue with: Mass Select/Multiselect Attribute value translater v1.0.1

I'm attempting to bulk add values to attributes using the following CSV data:

Maybe my CSV isn't formatted correctly? I'm not really sure, but I've not been able to find any documentation on this Utility. So I figured out the required columns by analyzing the code for massattr.php. (On an unrelated note, it kept saying Not Found when attempting to load the CSV so I had to add $params to initialize() on line 61).

But it ends up creating a duplicate entry in the eav_attribute_option_value table with the same Value.

I want to bulk add values for attributes, and specify both an "admin" value and a store view (ebay) value simultaneously. Or if I have to do this is in two imports, but how?

[apothegm]

I'd rather use the Mass Select Attribute value translater utility, but I also tried this with the regular Product importer by using the following CSV data:

It creates the new value for ebay_condition_id, but it doesn't seem to process the second row setting the ebay view value of 1500. Even if I separate them into different imports, it doesn't add the store view value of 1500.
According to this: http://wiki.magmi.org/index.php?title=Magmi_Behaviour_-_select_%26_multiselect_attributes
It should work.

I'm using latest version off sourceforge: 0.7.22

[T4ngml]

If it can help...

I've also been facing way too much duplicated entries in the attribute_option_value table and millions in catalog_product_attribute_pricing, causing the .ibdata file and import timing growing dangerously.

We finally discovered that duplicated entries were coming from unmanaged input values using special characters (such as ¾). As these values were not handled, "39¾" was becoming a new "39" entry, at each import.
Once these values were handled correctly, we could clean our tables selectively (only the problematic values), and get back to normal. Since, no more duplicated entries have been created, and the number of entries in tables keep reasonable.

Finally, if your database has been signifiticatibly growing, remember that truncate or delete data from tables won't drop your database weight. To get back to normal, you'll have to dump it, delete ibdata file and source it back.