Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PDF files not uploading with PHP 7 and Joomla 3.7.2 #16238

Closed
jen4web opened this issue May 24, 2017 · 81 comments
Closed

PDF files not uploading with PHP 7 and Joomla 3.7.2 #16238

jen4web opened this issue May 24, 2017 · 81 comments

Comments

@jen4web
Copy link

jen4web commented May 24, 2017

Steps to reproduce the issue

Go to: Content / Media

Click green Upload button, upload PDF file

Expected result

PDF file uploads without issue

Actual result

Error
Invalid mime type detected.

System information (as much as possible)

Site hosted at Rochen

Database Version 10.1.23-MariaDB
PHP Version 7.0.19
Web Server LiteSpeed
WebServer to PHP Interface litespeed
Joomla! Version Joomla! 3.7.2 Stable [ Amani ] 22-May-2017 09:46 GMT
Joomla! Platform Version Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT

I have disabled "check mime types" in the Media Options (click Options button from Media screen).

Legal MIME types: image/jpeg,image/gif,image/png,image/bmp,application/x-shockwave-flash,application/msword,application/excel,application/pdf,application/powerpoint,text/plain,application/x-zip

Legal extensions: bmp,csv,doc,gif,ico,jpg,jpeg,odg,odp,ods,odt,pdf,png,ppt,swf,txt,xcf,xls,BMP,CSV,DOC,GIF,ICO,JPG,JPEG,ODG,ODP,ODS,ODT,PDF,PNG,PPT,SWF,TXT,XCF,XLS

Additional comments

I have been uploading files through the Media Manager. JPG and PNG work, while PDF does not.

I reported this to Rochen, who confirmed that finfo_open and mime_content_type were installed, available, and working correctly.

Their response was this: "Well there seems to be compatibility issue with mime_content_type on Joomla version and PHP v7.0, If I switch the PHP version to v5.6 it is working fine.

It appears the patch provided isn't working on 3.7.1 and 3.7.2, due to mime_content_type. Thanks!"

Previously I was running PHP 7.0.19.

@ghost
Copy link

ghost commented May 24, 2017

Can't confirm, test using PHP 7.0.15 and 7.1.1.

System information

3.7.3-dev
Multilanguage Site
macOS Sierra, 10.12.5
Firefox 53 (64-bit)

MAMP 4.1.1

  • MySQLi 5.6.35

@C-Lodder
Copy link
Member

I cannot replicate this at all.

  • Joomla 3.7.3-dev
  • Wampserver
  • PHP 7.0.10

@Quy
Copy link
Contributor

Quy commented May 24, 2017

Cannot replicate:

  • PHP 7.0.19
  • Apache/Linux
  • Joomla! 3.7.2 Stable

@esedic
Copy link
Contributor

esedic commented May 24, 2017

I also couldn't replicate this issue, pdf uploaded without errors.
Joomla 3.7.2 with PHP 7.1.1

@zero-24 zero-24 self-assigned this May 24, 2017
@stevlam
Copy link

stevlam commented May 24, 2017

PHP 7.0.18
Joomla! 3.7.2 Stable
Linux

we're having this issue - "fileinfo.so" php extension was disabled in server.
enabled fileinfo php extension and all good.

maybe a more informative error message would help debuging issue adding something like "Could not detect the file mime type. Please check with your host if you have fileinfo PHP extension enabled.".

anyway with php fileinfo extension enabled cannot replicate the issue.

@jen4web
Copy link
Author

jen4web commented May 24, 2017

My fileinfo extension is enabled.

@JazParkyn
Copy link
Member

JazParkyn commented May 24, 2017

Confirmed issue on Rochen server:

PHP Built On Linux impress51.directrouter.com 3.10.0-427.36.1.lve1.4.43.el7.x86_64 #1 SMP Wed Mar 29 16:13:25 EDT 2017 x86_64
Database Version 10.1.23-MariaDB
Database Collation utf8_general_ci
Database Connection Collation utf8mb4_general_ci
PHP Version 7.0.18
Web Server Apache
WebServer to PHP Interface cgi-fcgi
Joomla! Version Joomla! 3.7.2 Stable [ Amani ] 22-May-2017 09:46 GMT
Joomla! Platform Version Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT
User Agent Mozilla/5.0 (Windows NT 10.0; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0

@Quy
Copy link
Contributor

Quy commented May 24, 2017

I have disabled "check mime types" in the Media Options (click Options button from Media screen).

@zero-24 Since Check MIME Types is disabled, Invalid mime type detected. shouldn't be checked/displayed. Right???

@jen4web
Copy link
Author

jen4web commented May 24, 2017

Update from Rochen --

After telling me that fileinfo and mime_content_type were enabled, they were able to get the PDF upload working under PHP 5.6. (In the control panel, go to PHP version and switch it.)

However, a second support person has come back to me after that change was made. He switched PHP 5.6 back to PHP 7.0. The PDF upload now works.

He says "I suppose it's possible that a needed PHP extension (like fileinfo) hadn't been selected previously, but the new defaults in the selector resolved the problem."

So, suggestion for other Rochen people with this issue:

a. log into cPanel
b. Choose "select PHP version" under "software" heading
c. See if fileinfo is checked.

If it IS checked, and you can't upload PDFs, then try:
a. Switch the version to 5.6 with the dropdown and apply the setting
b. Now switch back to PHP 7.0. Test to see if you can upload PDFs now.

My guess is that "fileinfo" is showing as checked when it's not really set.

@jen4web
Copy link
Author

jen4web commented May 24, 2017

Quy -- yes, that was my expectation for turning off the MIME type check. However, it had no effect on my ability/inability to upload a PDF.

Fix (???) is listed in the post before this one.

@Quy
Copy link
Contributor

Quy commented May 24, 2017

@jen4web My expectation too. The code has to be reworked to first check that Check MIME Types is enabled and not after the fact.

@zero-24
Copy link
Contributor

zero-24 commented May 24, 2017

@jen4web what error do you get when mime checking is disabled? As this should never be a option you should set.

If you try to upload a file on a server where we can not check or detect the mime type we don't allow uploading it.

The Check MIME Types check is about checking the actual mime type against the allowed mime type. If no mime type is detected this is not taken into account and upload is blocked.

@jen4web
Copy link
Author

jen4web commented May 24, 2017

My issue is resolved at this point. However, when it was broken, I tried uploading the PDF with mime checking enabled and disabled. Error message was the same:

Error
Invalid mime type detected.

@zero-24
Copy link
Contributor

zero-24 commented May 24, 2017

Sure we can improve that message. But this means "We can not detect any mime type for that file so we don't allow uploading it"

@Quy
Copy link
Contributor

Quy commented May 24, 2017

Here is the description for Check MIME Types:

Use MIME Magic or Fileinfo to attempt to verify files. Try disabling this if you get invalid mime type errors.

getMimeType() should only be executed if Check MIME Types is enabled.

@zero-24
Copy link
Contributor

zero-24 commented May 24, 2017

invalid mime type errors.

The problem is that you do not get "invalid mime type error" (i have already sayed that the message needs to be inproved) you get the "mime type detection is not working on your system" error. Which IMO should be fixed on server side and not just ignored by the CMS. As this is a server / configuration error and not a CMS error ;).

We maybe need a suggestion for a better error message in that case.

@zero-24
Copy link
Contributor

zero-24 commented May 24, 2017

Please take a look here: #16246 for a extended message.

@gwsdesk
Copy link

gwsdesk commented May 24, 2017

I am sorry but I cannot replicate this on our servers (PHP 7.1.14). Neither in default media manager nor in /TinyMCEJCE. I can upload pdf's either from frontend or admin without problems and if pdf's disabled (in media options) I do get the proper message (extension not supported)

However: The pdf's show in the file system with FTP (after upload in the media manager but they do not show after upload in the media manager itself in the administrator (they are uploaded as stated since they show in the file manager with ftp) That only happens when uploaded in this case with Tiny/JCE but they show when uploaded with/in the admin panel in the media manager itself (!)

so by using the media manager upload they show but when upload with any editor they don't show in the media manager in admin backend despite being listed in the file system (FTP)

@zero-24
Copy link
Contributor

zero-24 commented May 24, 2017

@gwsdesk

I am sorry but I cannot replicate this on our servers (PHP 7.1.14).

Sure the most of us can't as it looks like a server / hosting issue.

so by using the media manager upload they show but when upload with any editor they don't show in the media manager in admin backend despite being listed in the file system (FTP)

Please double check that. For the frontend image button this is expected (only images) and this is a longstanding behavior so nothing changed in 3.7.1/2

In the backend it is working fine for me:
image

@brianteeman
Copy link
Contributor

For reference I have notified Rochen of this issue and they are investigating

@mbabker
Copy link
Contributor

mbabker commented May 24, 2017

In general, things are working fine as long as the server is correctly configured. The issue we are running into it seems is that there are hosting environments (such as Rochen's) which are disabling features that are by default enabled in the baseline PHP distribution. So, we need to improve our handling a bit; though fileinfo is enabled by default since PHP 5.3.0 (source - https://secure.php.net/manual/en/fileinfo.installation.php), we can't rely on its presence so we need to get a little defensive in our coding. It's not much different than how we had to have some defensive coding around the use of parse_ini_file() and parse_ini_string().

@zero-24
Copy link
Contributor

zero-24 commented May 24, 2017

So, we need to improve our handling a bit

we can't rely on its presence so we need to get a little defensive in our coding.

Any suggestions?

@gwsdesk
Copy link

gwsdesk commented May 24, 2017

@zero-24 I do disagree with you on upload pdf. Please read what I wrote. I can upload with JCE for instance and it is successful. However the pdf won't show in the media manager but is in the file system.... See images
pdf_upload_ftp
pdf_upload1

@mbabker
Copy link
Contributor

mbabker commented May 24, 2017

That modal is hardcoded to only display images, even if the upload was successful it will never display PDF files.

@brianteeman
Copy link
Contributor

and that is NOT a new thing - it has always been that way

@gwsdesk
Copy link

gwsdesk commented May 24, 2017

I get you but why not? If mime-types are supported they should show?

@brianteeman
Copy link
Contributor

@dstncz based on all our tests it does sound like they are correct

@dustinklovera
Copy link

I'm having a hard time tracking this down. In addition to our site at SG we are also experiencing this issue with 3 additional sites on a separate container hosted at PowerVPS. One of which has barely any extra extensions (3.7.1 with Akeeba Backup, Brute Force Stop, JCE & Rokcandy).

I've tried disabling all the mentioned extensions and still get the mime error.

Related: I've confirmed that fileinfo is enabled on this container as well with the host.

Thought it might possibly be something with CXS, so fully disabled that. Still getting the error.

Any ideas?

@dustinklovera
Copy link

I was able to resolve the issue on both our containers. For the sites running 3.7.1 upgrading to 3.7.2 fixed it. For the SG site, we restored to a backup of 3.7.1 then re-upgraded to 3.7.2.

@cloakendagger
Copy link

Joomla 3.7.2 with PHP 7.1.1
Was able to replicate the issue and then solved it by enabling fileinfo.so

@hamsel42
Copy link

Joomla 3.8.0 with PHP 7.0.22 - same for PHP 7.1.7:

Seems like upgrading from Joomla! 3.7.2 to 3.8.0 has reintroduced the Invalid mime type problem.
phpinfo says version 7.0.22 and fileinfo enabled, yet an attempt to upload a PDF via Media Mgr fails with "Illegal or invalid mime type".

/Perscreen shot 2017-09-25 at 23 28 30

screen shot 2017-09-25 at 23 28 32screen shot 2017-09-25 at 23 28 34screen shot 2017-09-25 at 23 28 36


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/16238.

@hamsel42
Copy link

BTW:
"We use mime magic."


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/16238.

@simbus82
Copy link
Contributor

simbus82 commented Nov 30, 2017

Hi, i have similar problem.
Joomla 3.8.2, PHP 5.6 and php 7.0

I have mime, fileinfo, etc...

After trying to upload a PDF in media manager, even with check mime types OFF and PDF in excluded extension, even with application/octet-stream i receive this error:

0 - Invalid controller: name='file', format='html'

I can't upload with Docman too...

systeminfo-2017-11-30T14_47_16+00_00.txt

EDIT: with K2 Media Manager i can upload without any problem!

EDIT 2: If i try to upload with JCE File Browser i receive "The server returned an invalid JSON response."

@robbiejackson
Copy link

I think the problem could be alleviated by changing some of the code in the Media Helper canUpload() method, around lines 270 to 283 in libraries/src/Helper/MediaHelper.php in joomla 3.8.

Change:

if ($mime != false)
{
   :	[check MIME type is legal]
}
// We can't detect the mime type so it looks like an invalid file
else
{
	$app->enqueueMessage(\JText::_('JLIB_MEDIA_ERROR_WARNINVALID_MIME'), 'error');
	return false;
}
if (!\JFactory::getUser()->authorise('core.manage', $component))
{
	$app->enqueueMessage(\JText::_('JLIB_MEDIA_ERROR_WARNNOTADMIN'), 'error');
	return false;
}

to

if ($mime != false)
{
   :	[check MIME type is legal]
}
// We can't detect the mime type so it looks like an invalid file
else
{
	if (!\JFactory::getUser()->authorise('core.manage', $component))
	{
		$app->enqueueMessage(\JText::_('JLIB_MEDIA_ERROR_WARNNOTADMIN'), 'error');
		return false;
	}
}

This means that if for some reason the MIME type can't be ascertained, the code checks if the user has Media "core.manage" access before it disallows the upload. So users with sufficient privilege can still do their job, without having to relax the Restrict Uploads setting. (It would also fix this problem).

To my mind, this also fixes another inconsistency when the Restrict Uploads is set. When a user who doesn't have "core.manage" access tries to upload a non-image file – even though the file has a legal MIME type, the upload is still rejected because the user doesn't have "core.manage" privilege. Which doesn't in my opinion fit with the description of the Restrict Uploads setting "Restrict uploads for lower than manager users to just images if Fileinfo or MIME magic isn't installed.", in that it restricts it even if the system can determine the MIME type, and the file passes the legal MIME types check.

@ecoita
Copy link

ecoita commented Jan 25, 2018

Oddly enough, I've looked in many places and could not find answers that worked for me. Found this one below and it worked like a charm. Not sure why though...

Login to your Joomla Backend.
Go to The System Menu and select Global Configuration.
Under the component menus, select Media.
In the Media Section, scroll to Ignored Extensions
Type in the following text:   application/pdf,pdf,txt,text/plain

Try uploading a pdf, that is what fixed it for me.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/16238.

@Quy
Copy link
Contributor

Quy commented Mar 15, 2018

@zero-24 Can this be closed?

@hamsel42
Copy link

The work-around works, because it is negatively ignored as a media type. Since PDFs play an important role in SoHo site creations, e.g. for ideal communities, PDF as a mime type should be allowed per default. My recommendation would be for the Joomla core team to incorporate a fix into the next Joomla release to allow for PDFs.

@zero-24
Copy link
Contributor

zero-24 commented Mar 19, 2018

Thanks for your recomendation.

PDF has been enabend per default for serveral releases. But for obius reasons we don't overwrite your settings using update ;-)

This is more a issue with your server not beeing able to detect the file extension.

To me it is the expected behavior that if a extension cant bee detected the upload fails (this is the reason i wrote the code this way) but I'm happy to get another opinions. Technical this is a broken server config and well we can implement workarrounds but they will all compromise Security..

@carcam
Copy link

carcam commented Mar 20, 2018

In any case the setting "Check MIME Types" is not taken into account, so at least we should provide a fix for that, right?

@robbiejackson
Copy link

If the upload is always disallowed if the MIME type can't be determined, then in that case what does the config parameter Restrict Uploads mean? Its tooltip is "Restrict uploads for lower than manager users to just images if Fileinfo or MIME magic isn't installed".

I think there's an inconsistency between the tooltip description of this config parameter and the actual coded functionality, and it would be helpful to fix this.

@Quy
Copy link
Contributor

Quy commented Apr 12, 2018

@zero-24 I don't see Illegal MIME Types setting being utilized. Do you know why?

@ecoita
Copy link

ecoita commented Apr 12, 2018 via email

@zero-24
Copy link
Contributor

zero-24 commented Apr 12, 2018

@zero-24 I don't see Illegal MIME Types setting being utilized. Do you know why?

Yes because anything that is not allowed is illegal ;)

@Quy
Copy link
Contributor

Quy commented Apr 13, 2018

Yes because anything that is not allowed is illegal ;)

Then why have Illegal MIME Types setting?

Check MIME Types = Use MIME Magic or Fileinfo to try to verify files. Try disabling this if you get invalid mime type errors.

According to the above description, this setting can be used to disable the invalid mime type errors, but that is not possible since it is performed after getMimeType.

Shouldn't it be something like this?

if ($params->get('check_mime', 1))
{
        $mime = $this->getMimeType();
        ....
}

@robbiejackson
Copy link

robbiejackson commented Apr 13, 2018

Perhaps it's best to consider what the options should be, and ensure that the code then matches that. The fact that Fileinfo is now an inherent part of PHP has changed things a bit.

I've documented below what the options tooltips are, and what I believe the functionality is in the code, as a starting point. Worth checking I've got it right!

Restrict Uploads
parameter: restrict_uploads
tooltip: Restrict uploads for lower than manager users to just images if Fileinfo or MIME Magic isn't installed.
code: Restrict uploads for lower than manager users to just images

Check MIME Types
parameter: check_mime
tooltip: Use MIME Magic or Fileinfo to attempt to verify files. Try disabling this if you get invalid mime type errors.
code: Only relevant if Restrict Uploads is set to Yes. If Check MIME Types is Yes then it ensures that the MIME type also is valid (ie in addition to the file extension being valid). This applies for all users, including managers. A valid MIME type is one which is in the list of Legal MIME Types. Even if Check MIME Types is set to No, the system checks that the MIME Type can be determined.

Legal Image Extensions (File Types)
parameter: image_extensions
tooltip: Image extensions (file types) you are allowed to upload (comma separated). These are used to check for valid image headers.
code: Extensions (file types) you are allowed to upload (comma separated). [These don't have to be image extensions].

Ignored Extensions
parameter: ignore_extensions
tooltip: Ignored file extensions for MIME type checking and restricted uploads.
code: File extensions of files which any user will always be allowed to upload.

Legal MIME Types
parameter: upload_mime
tooltip: A comma separated list of legal MIME types to upload.
code: If Restrict Uploads is set to Yes and Check MIME Types is set to yes, then the MIME type is checked against this list, and the upload rejected (for all users, including managers) if the MIME type is not found in the list.

Illegal MIME Types
parameter: upload_mime_illegal
tooltip: A comma separated list of illegal MIME types to upload (blacklist).
code: used in com_attachments AttachmentsHelper::upload_file to block uploads.

In addition there is the question of what should happen if the system can't determine the MIME type. At the moment, this results in an error if Restrict Uploads is set to Yes, regardless of whether Check MIME types option is set to Yes or No.

@Quy
Copy link
Contributor

Quy commented Apr 13, 2018

Restrict Uploads
parameter: restrict_uploads
tooltip: Restrict uploads for lower than manager users to just images if Fileinfo or MIME Magic isn't installed.
code: Restrict uploads for lower than manager users to just images

It should include this condition Fileinfo or MIME Magic isn't installed.

Please test PR #20156 to show/hide settings relating to restrict uploads.

@robbiejackson
Copy link

robbiejackson commented Apr 14, 2018

I'm not sure how relevant that condition Fileinfo or MIME Magic isn't installed is any longer.
Looking at this page, MIME magic was removed in PHP 5.3, and Fileinfo is now included in standard PHP.

So I think that this is one reason why these tooltip descriptions would benefit from being updated.

@Quy
Copy link
Contributor

Quy commented Apr 14, 2018

Several people in this thread have mentioned that they are using "mime magic".

To me it is the expected behavior that if a extension cant bee detected the upload fails (this is the reason i wrote the code this way) but I'm happy to get another opinions. Technical this is a broken server config and well we can implement workarrounds but they will all compromise Security..

Can we agree with this and not implement workarounds? Which tooltip to rephrase and to what?

@robbiejackson
Copy link

Here are some suggestions as a starter.

Personally I would change some of the field names as well as the tooltips. I find it confusing having one field called Legal Extensions and another called Legal Image Extensions. I think that showing the latter only if Restrict Uploads is Yes will help to some extent.

Suggestions for field name changes:

Legal Image Extensions (File Types) - change to Unrestricted Extensions (File Types)
Legal MIME Types - change to Unrestricted MIME Types.

Suggestions for tooltip descriptions:

Restrict Uploads – If set to Yes, then users who have not got manager access are restricted to only a subset of the Legal Extensions which can be uploaded. They may only upload a file if its extension is in the specified Unrestricted Extensions or (if Check MIME Types is set to Yes) if its MIME type is in the specified Unrestricted MIME Types.

Unrestricted Extensions – If Restrict Uploads is Yes, then this is the list of file extensions which any user may upload.

Unrestricted MIME Types – If Restrict Uploads is Yes, then this is the list of MIME Types which any user may upload.

@joomla-cms-bot
Copy link

Set to "closed" on behalf of @Quy by The JTracker Application at issues.joomla.org/joomla-cms/16238

@Quy
Copy link
Contributor

Quy commented Jun 27, 2019

Please test PR #25351


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/16238.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests