New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import products: Import fails with a security notice for CSV files #22208

Closed
arunsathiya opened this Issue Dec 13, 2018 · 59 comments

Comments

Projects
None yet
@arunsathiya
Copy link

arunsathiya commented Dec 13, 2018

Describe the bug

When importing a products CSV file on the native WooCommerce products importer, it fails with a Sorry, this file type is not permitted for security reasons. notice.

To Reproduce

  • Start with a WooCommerce site
  • Visit {domain}/wp-admin/edit.php?post_type=product&page=product_importer
  • Select a CSV file from computer. Example file from WooCommerce's GitHub
  • Optionally choose to Update existing products
  • Click Continue

What is expected

The importer works

What happens instead

The import fails with a Sorry, this file type is not permitted for security reasons. notice.

Screenshots

screenshot 2018-12-13 at 12 39 55

Isolating the problem (mark completed items with an [x]):

  • I have deactivated other plugins and confirmed this bug occurs when only WooCommerce plugin is active.
  • This bug happens with a default WordPress theme active, or Storefront.
  • I can reproduce this bug consistently using the steps above.

WordPress Environment

``` ` ### WordPress Environment ###

Home URL: https://ethnic-pantropical.jurassic.ninja
Site URL: https://ethnic-pantropical.jurassic.ninja
WC Version: 3.5.2
Log Directory Writable: ✔
WP Version: 5.0.1
WP Multisite: –
WP Memory Limit: 256 MB
WP Debug Mode: –
WP Cron: ✔
Language: en_US
External object cache: –

Server Environment

Server Info: Apache/2.4.37 (Unix) OpenSSL/1.0.2g
PHP Version: 7.0.33 - We recommend using PHP version 7.2 or above for greater performance and security. How to update your PHP version
PHP Post Max Size: 1 GB
PHP Time Limit: 30
PHP Max Input Vars: 5000
cURL Version: 7.47.0
OpenSSL/1.0.2g

SUHOSIN Installed: –
MySQL Version: 5.7.24-0ubuntu0.16.04.1-log
Max Upload Size: 512 MB
Default Timezone is UTC: ✔
fsockopen/cURL: ✔
SoapClient: ✔
DOMDocument: ✔
GZip: ✔
Multibyte String: ✔
Remote Post: ✔
Remote Get: ✔

Database

WC Database Version: 3.5.2
WC Database Prefix: wp_9fd08c7a0b_
MaxMind GeoIP Database: ✔
Total Database Size: 2.63MB
Database Data Size: 1.76MB
Database Index Size: 0.87MB
wp_9fd08c7a0b_woocommerce_sessions: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_woocommerce_api_keys: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_woocommerce_attribute_taxonomies: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_woocommerce_downloadable_product_permissions: Data: 0.02MB + Index: 0.05MB
wp_9fd08c7a0b_woocommerce_order_items: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_woocommerce_order_itemmeta: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_woocommerce_tax_rates: Data: 0.02MB + Index: 0.06MB
wp_9fd08c7a0b_woocommerce_tax_rate_locations: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_woocommerce_shipping_zones: Data: 0.02MB + Index: 0.00MB
wp_9fd08c7a0b_woocommerce_shipping_zone_locations: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_woocommerce_shipping_zone_methods: Data: 0.02MB + Index: 0.00MB
wp_9fd08c7a0b_woocommerce_payment_tokens: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_woocommerce_payment_tokenmeta: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_woocommerce_log: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_commentmeta: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_comments: Data: 0.02MB + Index: 0.08MB
wp_9fd08c7a0b_links: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_options: Data: 1.08MB + Index: 0.05MB
wp_9fd08c7a0b_postmeta: Data: 0.09MB + Index: 0.03MB
wp_9fd08c7a0b_posts: Data: 0.09MB + Index: 0.06MB
wp_9fd08c7a0b_termmeta: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_terms: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_term_relationships: Data: 0.02MB + Index: 0.02MB
wp_9fd08c7a0b_term_taxonomy: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_usermeta: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_users: Data: 0.02MB + Index: 0.05MB
wp_9fd08c7a0b_wc_download_log: Data: 0.02MB + Index: 0.03MB
wp_9fd08c7a0b_wc_webhooks: Data: 0.02MB + Index: 0.02MB

Post Type Counts

attachment: 21
customize_changeset: 1
feedback: 1
jetpack_migration: 2
jp_img_sitemap: 1
jp_sitemap: 1
jp_sitemap_master: 1
nav_menu_item: 2
page: 7
post: 29
product: 12
revision: 17
wp_block: 1

Security

Secure connection (HTTPS): ✔
Hide errors from visitors: ✔

Active Plugins (4)

Companion Plugin: by Osk – 1.6
Jetpack by WordPress.com: by Automattic – 6.8.1
WooCommerce Blocks: by Automattic – 1.2.0
WooCommerce: by Automattic – 3.5.2

Settings

API Enabled: –
Force SSL: –
Currency: GBP (£)
Currency Position: left
Thousand Separator: ,
Decimal Separator: .
Number of Decimals: 2
Taxonomies: Product Types: external (external)
grouped (grouped)
simple (simple)
variable (variable)

Taxonomies: Product Visibility: exclude-from-catalog (exclude-from-catalog)
exclude-from-search (exclude-from-search)
featured (featured)
outofstock (outofstock)
rated-1 (rated-1)
rated-2 (rated-2)
rated-3 (rated-3)
rated-4 (rated-4)
rated-5 (rated-5)

WC Pages

Shop base: Page not set
Cart: Page not set
Checkout: Page not set
My account: Page not set
Terms and conditions: Page not set

Theme

Name: Photos
Version: 1.0.1
Author URL: https://wordpress.com/themes/
Child Theme: – If you are modifying WooCommerce on a parent theme that you did not build personally we recommend using a child theme. See: How to create a child theme
WooCommerce Support: Not declared

Templates

Overrides: –
`

</details>
@arunsathiya

This comment has been minimized.

Copy link

arunsathiya commented Dec 13, 2018

As a workaround, installing https://wordpress.org/plugins/disable-real-mime-check/ and attempting to import again does not show the error. h/t @hafizrahman

@kloon

This comment has been minimized.

Copy link
Member

kloon commented Dec 13, 2018

Seems related to the WP 5.0.1 update which now enforces mime type checking. Working on a fix.

@kloon kloon added the bug label Dec 13, 2018

@kloon kloon added this to the 3.5.3 milestone Dec 13, 2018

@kloon

This comment has been minimized.

Copy link
Member

kloon commented Dec 13, 2018

After some further investigation, this seems to be a bug in WP core and not WooCommerce. CSV is an allowed type, but the mime type can be text/plain as well as text/csv and WP only caters for the one.

Opened a tract ticket with a patch here https://core.trac.wordpress.org/ticket/45615

@kloon kloon removed the bug label Dec 13, 2018

@kloon kloon removed this from the 3.5.3 milestone Dec 13, 2018

@kloon kloon referenced this issue Dec 13, 2018

Closed

Add text/plain mime type to csv importer #22209

5 of 6 tasks complete
@kloon

This comment has been minimized.

Copy link
Member

kloon commented Dec 13, 2018

Workaround in #22208

@kloon kloon added this to the 3.5.3 milestone Dec 13, 2018

@batesy87

This comment has been minimized.

Copy link

batesy87 commented Dec 13, 2018

Seems related to the WP 5.0.1 update which now enforces mime type checking. Working on a fix.

I have had this happen today on two sites running WP 4.9.9 and 4.8.8 also

@kloon

This comment has been minimized.

Copy link
Member

kloon commented Dec 13, 2018

I have had this happen today on two sites running WP 4.9.9 and 4.8.8 also

Yes, 5.0.1 was a security fix which means they backported the fixes to older versions of WordPress as well, so you will see it in those releases too.

@batesy87

This comment has been minimized.

Copy link

batesy87 commented Dec 13, 2018

A better workaround is this

add_filter("mime_types", "add_csv_plain");
function add_csv_plain($mime_types)
{

    unset($mime_types['txt|asc|c|cc|h|srt']);
    $mime_types['txt|asc|c|cc|h|srt|csv'] = 'text/plain';

    return $mime_types;
}

add_filter("woocommerce_csv_product_import_valid_filetypes", "add_csv_plain_woocommerce");
function add_csv_plain_woocommerce()
{
    return [
        'txt|csv' => 'text/plain',
        'csv' => 'text/csv',
    ];
}

However I believe if a csv file is uploaded with the mime type of text/csv it would still fail because now we are saying that csv should = text/plain

I think Wordpress need to add something into the real mime type check to allow csv to be either text/plain or text/csv

@homelylittletouch

This comment has been minimized.

Copy link

homelylittletouch commented Dec 13, 2018

As a workaround, installing https://wordpress.org/plugins/disable-real-mime-check/ and attempting to import again does not show the error. h/t @hafizrahman

This has worked for me to import products, but the images aren't working. They are in my media folder on WP and I tried the file name, the file name.jpg, the full URL and wp-content/uploads/2018/10/N0006-1.jpg
None of these have worked so all 703 products are without pictures! Any ideas please?

@dougaitken

This comment has been minimized.

Copy link
Member

dougaitken commented Dec 13, 2018

Hey @homelylittletouch

This has worked for me to import products, but the images aren't working

I've just tested with two CSV files, one product with a remote image and one with an image already at {site}/wp-content/uploads/2018/12/beanie.jpg and both worked.

I'd retry the CSV with the Product ID and Images columns and "update existing products". If the images aren't attaching, this could be another issue at play.

@yukikatayama

This comment has been minimized.

Copy link

yukikatayama commented Dec 13, 2018

Another report in hc-8801734

@homelylittletouch

This comment has been minimized.

Copy link

homelylittletouch commented Dec 14, 2018

Hey @homelylittletouch

This has worked for me to import products, but the images aren't working

I've just tested with two CSV files, one product with a remote image and one with an image already at {site}/wp-content/uploads/2018/12/beanie.jpg and both worked.

I'd retry the CSV with the Product ID and Images columns and "update existing products". If the images aren't attaching, this could be another issue at play.

Thank you, this has worked now with the original file.

@dougaitken

This comment has been minimized.

Copy link
Member

dougaitken commented Dec 14, 2018

Interactions I've had around this:

1643312-zen
1643577-zen
1642767-zen
1643372-zen
1643225-zen

@homelylittletouch

This comment has been minimized.

Copy link

homelylittletouch commented Dec 14, 2018

Not sure if this is just me but since this new update, when I was finally able to import my products and pictures, nothing seems to be working properly. Menus are not updating properly, my category slider has gone off the page and is not showing the categories I selected. Is this a coincidence or all related? Are WP working on a new version to fix the bugs; I'm apprehensive to advertise a website that isn't working properly!

@ruphu5

This comment has been minimized.

Copy link

ruphu5 commented Dec 14, 2018

I just experienced this issue as well. My WP/WC installation did an update by itself to 4.9.9 and now I can not import my products through .csv anymore.

What is the current situation on this matter? Should I update further on to 5.0.1? I am a little scared, that it gets even worse....

As I read, the workaround with the MIME plugin will cause problems with showing the images... So I do not know, what to do now.

// edit:
my workaround by now is, to ftp upload the csv and then use that one for importing new products

@avlisesac

This comment has been minimized.

Copy link

avlisesac commented Dec 14, 2018

@ruphu5 Would you mind explaining that process in a little more detail? I'd be interested to try that option myself.

@ruphu5

This comment has been minimized.

Copy link

ruphu5 commented Dec 14, 2018

@ruphu5 Would you mind explaining that process in a little more detail? I'd be interested to try that option myself.

When importing new products, you have the option to use an uploaded .csv file on your webspace.
Login -> Products -> Import -> "Show advanced options" -> set the path to your .csv file (e.g. 'wp-content/uploads/your-new-products.csv') -> Press Continue

@avlisesac

This comment has been minimized.

Copy link

avlisesac commented Dec 14, 2018

@ruphu5 Would you mind explaining that process in a little more detail? I'd be interested to try that option myself.

When importing new products, you have the option to use an uploaded .csv file on your webspace.
Login -> Products -> Import -> "Show advanced options" -> set the path to your .csv file (e.g. 'wp-content/uploads/your-new-products.csv') -> Press Continue

Thanks, I'll give that a try!

@claudiulodro

This comment has been minimized.

Copy link
Contributor

claudiulodro commented Dec 14, 2018

The WP core team is aware of the issue, and will have this issue fixed. (Thanks for submitting the core patch @kloon!)

If it's not fixed in WordPress core by the time WooCommerce 3.5.3 goes out, we'll add a workaround in WC 3.5.3.

In the meantime, the easiest workaround is to rename your CSV file from foo.csv to foo.txt, and everything should continue working.

@jmzolezzi

This comment has been minimized.

Copy link

jmzolezzi commented Dec 20, 2018

It's not ideal, but we've been using the following setting at wp-config.php to sort this, while an update and fix is released.

define( 'ALLOW_UNFILTERED_UPLOADS', true );

@Linuxhombre

This comment has been minimized.

Copy link

Linuxhombre commented Dec 20, 2018

I tried this @jmzolezzi and it unfortunately did not assist.

Any known interim workarounds? We've tried everything down to the MU plugin and this is totally a CORE issue that we expect to be deployed by WordPress in the next 10-20 hours.

@Linuxhombre

This comment has been minimized.

Copy link

Linuxhombre commented Dec 20, 2018

WP Version: 5.0.1

FYI - this is now present in 5.0.2 as well, released last night.

@dougaitken

This comment has been minimized.

Copy link
Member

dougaitken commented Dec 21, 2018

As Claudiu mentioned:

In the meantime, the easiest workaround is to rename your CSV file from foo.csv to foo.txt, and everything should continue working.

I tested this method earlier and it worked.

When a fix is released, if it is by us, it will be on https://woocommerce.wordpress.com and included in the release notes.

If it is by the WordPress folks, then it will again be in the release notes.

@AaronBowie

This comment has been minimized.

Copy link

AaronBowie commented Dec 21, 2018

A quick workaround is to use the Disable Real MIME Check plugin. I too had this issue today, I needed a quick workaround which I used the plugin for.

@Linuxhombre

This comment has been minimized.

Copy link

Linuxhombre commented Dec 21, 2018

Hi @AaronBowie -
This plugin did not work for 5.0.1 :(

What version do you have installed of WordPress?

Thanks..
JF

@AaronBowie

This comment has been minimized.

Copy link

AaronBowie commented Dec 21, 2018

@Linuxhombre Hi,

I used WP 5.0.1 and Woo 3.5.1 I downgraded both WP and Woo to get this to work.

@Linuxhombre

This comment has been minimized.

Copy link

Linuxhombre commented Dec 21, 2018

Hi @AaronBowie - unfortunately that really isn't an option for us.

Any known workarounds right now?

Cheers!
J

@Sevun11

This comment has been minimized.

Copy link

Sevun11 commented Dec 26, 2018

It has been a long time. Do we have a timeline for next WP/WC release? Will it fix this? If not I'll downgrade to WP 4.9.8.

Thanks

@rrennick

This comment has been minimized.

Copy link
Collaborator

rrennick commented Dec 26, 2018

Do we have a timeline for next WP/WC release?

Probably not for at least another couple weeks.

@adamleone

This comment has been minimized.

Copy link

adamleone commented Dec 26, 2018

Another issue reported here zen-1669580.

Installing Disable Real MIME Check fixed the issue.

@Sevun11

This comment has been minimized.

Copy link

Sevun11 commented Dec 26, 2018

Yep thanks for your answer. I had as well same message(security) while importing media. Waiting a couple of weeks more then.

@Linuxhombre

This comment has been minimized.

Copy link

Linuxhombre commented Dec 27, 2018

Yes I made every permutation of changes, but reinstalling the WP core was the best temporary solution.

Quite a big bug to wait several weeks for. Maybe WooCommerce can make a developer-side announcement, as I'm sure not all are in-the-know regarding this current (and very strange) issue.

Thanks Team!
J

@mlaetitia

This comment has been minimized.

Copy link

mlaetitia commented Dec 28, 2018

Another issue reported here: 8103324-hc . Installing the plugin disable-real-mime-check worked!

@mwendasam

This comment has been minimized.

Copy link

mwendasam commented Dec 28, 2018

Installing the Disable Real MIME Check Plugin solved the Issue.

@rrennick

This comment has been minimized.

Copy link
Collaborator

rrennick commented Dec 28, 2018

Quite a big bug to wait several weeks for.

It's a bug in WordPress that affects any plugin that uses uploaded CSVs vs being specific to WooCommerce. Since there is already a plugin available to use as a work around the best choice for affected plugins is to give WP core an opportunity to fix it.

@gugaalves

This comment has been minimized.

Copy link

gugaalves commented Dec 28, 2018

Another issue reported here: 8824694-hc, mentioned plugin made it work :)

@kriskarkoski

This comment has been minimized.

Copy link

kriskarkoski commented Jan 2, 2019

Another in 1683286-zen that the plugin workaround worked for as well.

@mkastler

This comment has been minimized.

Copy link

mkastler commented Jan 2, 2019

Just adding that renaming to .txt did not work, but the disable-real-mime-check plugin did resolve as a workaround. Per best practices, I'm planning on keeping the plugin deactivated except when needed for imports.

Shout out to the others who suggested this, saved me some hours and frustration for sure!!

@Linuxhombre

This comment has been minimized.

Copy link

Linuxhombre commented Jan 2, 2019

@dromero20

This comment has been minimized.

Copy link

dromero20 commented Jan 4, 2019

Another report in 9237290-hc

@madeincosmos

This comment has been minimized.

Copy link
Contributor

madeincosmos commented Jan 8, 2019

Also reported in 1698618-zen

@claudiulodro

This comment has been minimized.

Copy link
Contributor

claudiulodro commented Jan 8, 2019

This should be fixed in the WP 5.0.3 release going out tomorrow.

@mikejolley

This comment has been minimized.

Copy link
Member

mikejolley commented Jan 9, 2019

No changes needed our end - wait for the WP update.

@rorybot

This comment has been minimized.

Copy link

rorybot commented Jan 17, 2019

Still persists for me. Followed @arunsathiya's suggestion to disable mime check.

@arunsathiya

This comment has been minimized.

Copy link

arunsathiya commented Jan 17, 2019

@rorybot Is that happening on WordPress 5.0.3 for you? I tested now and the upload works fine for me. You might also want to check if that happens with any related/all plugins disabled - essentially test on a vanilla install of WooCommerce.

@jfacemyer

This comment has been minimized.

Copy link

jfacemyer commented Jan 18, 2019

I can confirm that this is still an issue for me as well with 5.0.3. I've disabled most plugins on my test site. I can't get it to work with CSVs in UTF8 from LibreOffice, not even if I rename them to .txt. I ended up putting the file on my server for import, but that's a huge pain unless you know it's exactly right the first time (well, and even then it's a pain.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment