model rule - 'file' extensions seem not 100% work

[ScottHuangZL]

Dear Yii2 Team,

I try to upload file with below model. It is work tofilter xls/zip and so on, however, the extensions limitation failed for outlook .msg files.
It allow select .msg file, but failed to pass validate model, and then cannot upload.

Can you help to look into for this issue? Thanks.

public function rules()
    {
        return [
            [['excelFile'], 'file','skipOnEmpty' => false
                ,'extensions' => 'xls, xlsx, xlsm, msg, zip, 7z, rar'
                ,'maxSize'=>1024*1000 //1M
//                , 'mimeTypes' =>[ 'xls' => 'application/vnd.ms-excel'] //text/plain, image/png
            ],
        ];
    }

[samdark]

It's a known issue because for some files it's not possible to detect type based on contents. For these you can validate against plaintext and check file extension. Not reliable but would work at least.

[ScottHuangZL]

No, I not mention the mimeTypes, you see, I not turn on the 'mimeTypes'.
I just talk about the extensions itself.

For example:
'extensions' => 'xls, xlsx, xlsm, msg, zip, 7z, rar'

I can select xxx.msg to upload at 1st step, but $model->validate() step return false and warning me only can upload 'xls, xlsx, xlsm, msg, zip, 7z, rar' files.

Can you double check?

[SilverFire]

Can you post a sample of .msg file, please?

[ScottHuangZL]

.msg file is outlook emails. Usually, I can copy one email from outlook and save to desktop as .msg file.
I cannot attached my company email sample to you due to regulation.

But I think you can rename any xxx.txt file to xxx.msg to testing.
Thanks.

[SilverFire]

Okay, I got your problem. File extension msg is not described in yii2\helpers\mimeTypes.php, so validator treats your msg file as unacceptable.

I can offer you two solutions:

• create your own mimeTypes.php and merge ['msg' => 'text/plain'] with contents of default one. Then configure FileHelper and set the mimeMagicFile pointing to your customized file.
• in model rule of file validation set 'checkExtensionByMimeType' => false. Be careful about this, as @samdark mentioned, it is not reliable but would work at least.

[ScottHuangZL]

Can you add .msg type into mimeTypes?
Outlook email is popular, thanks.

[ScottHuangZL]

I select option

'checkExtensionByMimeType' => false

Can Yii2 team at least provide different error message in case I not add above code?
It should be not ext error, it is mimeType checking error.

Or, set that flag default as false.

Or, handle .msg type too, since as I mentioned, it is outlook email, popular. Thanks.

[samdark]

We're importing Apache mime types: https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types. There's no plan to add more types except if the file is updated by Apache project.

[ScottHuangZL]

Ok, thanks.

[luke-]

What do you think, if we only validate the mimetype of the extension, if we find a corresponding entry for it in the magicfile?

e.g. add 'FileHelper::getMimeTypeByExtension('test.' . $extension) !== null &&'
here https://github.com/yiisoft/yii2/blob/master/framework/validators/FileValidator.php#L402

[samdark]

@luke- is the problem you're trying to solve the same as the initial problem indicated in this issue?

[luke-]

@samdark yes, we just had the same issue with ".msg" files.
I think it should be sufficient to check the mime type of the file only if it's known (listed in mime type file).

Iust wanted to hear your opinion about this, here is our "fix":
humhub/humhub@49ee6ff#diff-b6fba767073a31f8509978169968c22bR137

[samdark]

@luke- while this mode of checking is convenient, it potentially may cause security issues if either using checkExtensionByMimeType = false or falling back to this mode as you are suggesting so I'd not have it in the framework.

[luke-]

@samdark In our case I want both, best possible security (if the mime type is known --> checkExtensionByMimeType), but also the support of (still) unknown file/mime types. But I understand that you don't want to have this behavior in the framework. Anyway, thanks for the clarification and also thanks for the good work!