attachment name is broken if it is in utf8 and longer than 24 characters

[z-eos]

attachment name is broken if it is utf8 and longer than 24 characters

resources

RC v.1.3.1, v.1.3.3
PHP v.5.6.30
imap4d (GNU Mailutils) 3.4

attached file name expected

абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx

attached file name provided by RC

Ð°Ð±Ð²Ð³Ð´ÐµÑ Ð¶Ð·Ð¸ÐºÐ»Ð¼Ð½Ð¾Ð¿Ñ Ñ Ñ Ñ Ñ Ñ Ñ ÑF#F'F+F_F3F7F;F_¹ ½ .docx

RC config

$config['mime_param_folding'] = 1;

original message fragment:

X-Envelope-Date: Tue Nov 14 09:16:53 2017
X-Envelope-Sender: jdoe@foo.bar

This is a multi-part message in MIME format.
--------------5BCCBC62D02AE2E4D591BEA1
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit



--------------5BCCBC62D02AE2E4D591BEA1
Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;
 name="=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE?=
 =?UTF-8?B?0YXRhtGH0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?="
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename*0*=utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7;
 filename*1*=%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82;
 filename*2*=%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C;
 filename*3*=%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78

UEsDBBQABgAIAAAAIQDfpNJsWgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIo
oAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

telnet session fragment

A0007 UID FETCH 6 (BODY.PEEK[2.MIME])
* 6 FETCH (UID 6 BODY[2.MIME] {565}
Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;
 name="=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE?=
 =?UTF-8?B?0YXRhtGH0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?="
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename*0*=utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7;
 filename*1*=%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82;
 filename*2*=%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C;
 filename*3*=%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78

)
A0007 OK UID FETCH Completed
A0008 UID FETCH 6 (BODY.PEEK[2])
* 6 FETCH (UID 6 BODY[2] {16196}
UEsDBBQABgAIAAAAIQDfpNJsWgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIo
oAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

RC debug fragment

Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: * 6 FETCH (UID 6 BODY[2.MIME] {565}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;^M  name="=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE?=^M  =?UTF-8?B?0YXRhtGH0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?="^M Content-Transfer-Encoding: base64^M Content-Disposition: attachment;^M  filename*0*=utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7;^M  filename*1*=%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82;^M  filename*2*=%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C;^M  filename*3*=%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78^M ^M
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: )
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: A0008 OK UID FETCH Completed
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] C: A0009 UID FETCH 6 (BODY.PEEK[2])
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: * 6 FETCH (UID 6 BODY[2] {16196}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: UEsDBBQABgAIAAAAIQDfpNJsWgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIo

miscelaneous

uudeview decodes correctly

uudeview 1510651013.M946286P86037Q0.relay.xx\:2,RS 

Loaded from 1510651013.M946286P86037Q0.relay.xx:2,RS: 'test' (test): абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx part 1   Base64
                                                                      
Found 'абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx' State 16 Base64 Parts 1 OK

  -rw-r--r-- абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx is OK   [d] (?=help)

[alecpl]

So, the message was created by Roundcube and you think it is malformed, or the message was created by some other client and Roundcube displays it wrong? Roundcube trusts BODYSTRUCTURE response, so check this in your debug log. I think it's more likely a bug on imap server side.

[z-eos]

any MUA (thunderbird, K9, roundcube, default android MUA were tested) sending attachment as I described, causing RC to break attachment name

as for the server, it is not changing the message itself, but I'll check that as you suggest

[z-eos]

the only difference I see is how RC receives name ... by chunks, while in telnet connection it comes as single piece \n delimited ...

telnet session

A0006 UID FETCH 6 (UID RFC822.SIZE FLAGS INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])
* 6 FETCH (UID 6 RFC822.SIZE 18020 FLAGS (\Seen) INTERNALDATE "14-Nov-2017 09:16:53 +0000" BODYSTRUCTURE (("text" "plain" ("charset" "utf-8" "format" "flowed") NIL NIL "7bit" 2 1 NIL NIL NIL)("application" "vnd.openxmlformats-officedocument.wordprocessingml.document" ("name" {118}
=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?UTF-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=) NIL NIL "base64" 16196 NIL ("attachment" ("filename" {118}
=?utf-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?utf-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=)) NIL) "mixed" ("boundary" "------------5BCCBC62D02AE2E4D591BEA1") NIL "ru") BODY[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)] {348}
RETURN-PATH: <jdoe@foo.bar>
TO: jdoe@foo.bar
CC: jdoe2@foo2.bar
FROM: "jdoe@foo.bar" <jdoe@foo.bar>
SUBJECT: test
MESSAGE-ID: <8c1d39d7-5fef-8740-0607-b6f22fb3e742@xprmail.org>
DATE: Tue, 14 Nov 2017 11:16:17 +0200
CONTENT-TYPE: multipart/mixed;
 boundary="------------5BCCBC62D02AE2E4D591BEA1"

)
A0006 OK UID FETCH Completed

RC debug

Nov 15 23:14:57 rc roundcube: <qekqslgo> [2BAC] C: A0007 UID FETCH 6 (UID RFC822.SIZE FLAGS INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])
Nov 15 23:14:57 rc roundcube: <qekqslgo> [2BAC] S: * 6 FETCH (UID 6 RFC822.SIZE 18020 FLAGS (\Seen) INTERNALDATE "14-Nov-2017 09:16:53 +0000" BODYSTRUCTURE (("text" "plain" ("charset" "utf-8" "format" "flowed") NIL NIL "7bit" 2 1 NIL NIL NIL)("application" "vnd.openxmlformats-officedocument.wordprocessingml.document" ("name" {118}
Nov 15 23:14:57 rc roundcube: <qekqslgo> [2BAC] S: =?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?UTF-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: ) NIL NIL "base64" 16196 NIL ("attachment" ("filename" {118}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: =?utf-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?utf-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: )) NIL) "mixed" ("boundary" "------------5BCCBC62D02AE2E4D591BEA1") NIL "ru") BODY[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)] {348}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: RETURN-PATH: <jdoe@foo.bar>^M TO: jdoe@foo.bar^M CC: jdoe2@foo2.bar^M FROM: "jdoe@foo.bar" <jdoe@foo.bar>^M SUBJECT: test^M MESSAGE-ID: <8c1d39d7-5fef-8740-0607-b6f22fb3e742@xprmail.org>^M DATE: Tue, 14 Nov 2017 11:16:17 +0200^M CONTENT-TYPE: multipart/mixed;^M  boundary="------------5BCCBC62D02AE2E4D591BEA1"^M ^M
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: )
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: A0007 OK UID FETCH Completed
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] C: A0008 UID FETCH 6 (BODY.PEEK[2.MIME])

[alecpl]

I'm not sure it's related, but my imap server returns also this:

("FILENAME*" {214}utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78)

but if I comment out "filename" handling, it's still decoded correctly. So, I suppose the headers are encoded correctly. Could you provide a complete sample message?

[z-eos]

sample message is here

[alecpl]

The sample works for me. So, more investigation needed. I'm using PHP 7.1, but I wouldn't say it's PHP's fault. Could you provide BODYSTRUCTURE response from your IMAP server for the same message?

[z-eos]

here it is

A0005 OK [READ-WRITE] SELECT Completed
A0006 UID FETCH 9 (UID RFC822.SIZE FLAGS INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])
* 9 FETCH (UID 9 RFC822.SIZE 348941 FLAGS (\Seen) INTERNALDATE "16-Nov-2017 07:58:31 +0000" BODYSTRUCTURE (("text" "plain" ("charset" "US-ASCII" "format" "flowed") NIL NIL "7bit" 37 3 NIL NIL NIL)("application" "pdf" ("name" {216}
=?UTF-8?B?Um91bmRjdWJlINC/0YDQuNC80LXRgCDQsdC40YLQvtCz0L4g0LjQvNC10L3QuCD?=  =?UTF-8?B?QstC70L7QttC10L3QuNGPINC00LvRjyDQv9GA0LjQvNC10YDQsCDRgtC+0LPQvi?=  =?UTF-8?B?DQutCw0Log0L7QvdC+INC10YHRgtGMINCx0YvRgtGMLnBkZg==?=) NIL NIL "base64" 346692 NIL ("attachment" ("filename" {216}
=?UTF-8?B?Um91bmRjdWJlINC/0YDQuNC80LXRgCDQsdC40YLQvtCz0L4g0LjQvNC10L3QuCD?=  =?UTF-8?B?QstC70L7QttC10L3QuNGPINC00LvRjyDQv9GA0LjQvNC10YDQsCDRgtC+0LPQvi?=  =?UTF-8?B?DQutCw0Log0L7QvdC+INC10YHRgtGMINCx0YvRgtGMLnBkZg==?= "size" "253352")) NIL) "mixed" ("boundary" "=_742fc19fa7c58c17da30ccc018454efc") NIL NIL) BODY[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)] {308}
RETURN-PATH: <zeus@ibs.dn.ua>
CONTENT-TYPE: multipart/mixed;
 boundary="=_742fc19fa7c58c17da30ccc018454efc"
DATE: Thu, 16 Nov 2017 09:58:01 +0200
FROM: Zeus Panchenko <zeus@ibs.dn.ua>
TO: hostmaster@ibs.dn.ua
SUBJECT: attachment name broken
MESSAGE-ID: <7e9465080b75b81e2fd78de7ded5082a@ibs.dn.ua>

)
A0006 OK UID FETCH Completed
A0007 UID FETCH 9 (BODY.PEEK[2.MIME])
* 9 FETCH (UID 9 BODY[2.MIME] {1041}
Content-Transfer-Encoding: base64
Content-Type: application/pdf;
 name="=?UTF-8?Q?Roundcube_=D0=BF=D1=80=D0=B8=D0=BC=D0=B5=D1=80_=D0=B1=D0?=
 =?UTF-8?Q?=B8=D1=82=D0=BE=D0=B3=D0=BE_=D0=B8=D0=BC=D0=B5=D0=BD=D0=B8_=D0?=
 =?UTF-8?Q?=B2=D0=BB=D0=BE=D0=B6=D0=B5=D0=BD=D0=B8=D1=8F_=D0=B4=D0=BB=D1?=
 =?UTF-8?Q?=8F_=D0=BF=D1=80=D0=B8=D0=BC=D0=B5=D1=80=D0=B0_=D1=82=D0=BE=D0?=
 =?UTF-8?Q?=B3=D0=BE_=D0=BA=D0=B0=D0=BA_=D0=BE=D0=BD=D0=BE_=D0=B5=D1=81?=
 =?UTF-8?Q?=D1=82=D1=8C_=D0=B1=D1=8B=D1=82=D1=8C=2Epdf?="
Content-Disposition: attachment;
 filename*0*=UTF-8''Roundcube%20%D0%BF%D1%80%D0%B8%D0%BC%D0%B5%D1%80%20%D0;
 filename*1*=%B1%D0%B8%D1%82%D0%BE%D0%B3%D0%BE%20%D0%B8%D0%BC%D0%B5%D0%BD;
 filename*2*=%D0%B8%20%D0%B2%D0%BB%D0%BE%D0%B6%D0%B5%D0%BD%D0%B8%D1%8F%20;
 filename*3*=%D0%B4%D0%BB%D1%8F%20%D0%BF%D1%80%D0%B8%D0%BC%D0%B5%D1%80%D0;
 filename*4*=%B0%20%D1%82%D0%BE%D0%B3%D0%BE%20%D0%BA%D0%B0%D0%BA%20%D0%BE;
 filename*5*=%D0%BD%D0%BE%20%D0%B5%D1%81%D1%82%D1%8C%20%D0%B1%D1%8B%D1%82;
 filename*6*=%D1%8C.pdf;
 size=253352

)
A0007 OK UID FETCH Completed

[alecpl]

The second response is evidently malformed by the server.

So, what's going on here. Roundcube sees that there are filename* "fields" in the header. In such case it's fetching complete header and uses it to get the attachment name. Maybe we could detect that and fallback to the name specified in BODYSTRUCTURE response.

[z-eos]

sorry much, I didn't catch in sight that but it was copy-paste error
I corrected the previous message
now all fields `filename' are where they have to be ...

[z-eos]

was it helpful?

[alecpl]

Now I have no clue. While obviously your BODYSTRUCTURE response is different than mine, I'm talking about "filename" part, I'm not able to reproduce the issue even if I change the code to use your bodystructure.

What is your locale? What default character set (Preferences > Displaying Messages > Advanced Options)? Did you try different browser? Did you try different skin? Did you try with disabled all plugins? Do you have php-mbstring and/or php-iconv installed? Where exactly do you see the broken text, in message preview or when downloading the file?

[z-eos]

• locale is UTF8

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8

• default charset is UTF8
• browsers tested: FireFox (unix, windows), Chrome (windows)
• Classic and Larry skins produce the same result
• plugins disabled have no effect, though About still shows jquery and filesystem_attachments enabled even with $config['plugins'] = array(); in config
• php-mbstring and php-iconv both are installed
• I see the broken text, in both, message preview and file download dialogue

[alecpl]

I should have find this earlier. Anyway, there's a bug in handling names encoded this way:

=?UTF-8?B?Um91bmRjdWJlINC/0YDQuNC80LXRgCDQsdC40YLQvtCz0L4g0LjQvNC10L3QuCD?=
=?UTF-8?B?QstC70L7QttC10L3QuNGPINC00LvRjyDQv9GA0LjQvNC10YDQsCDRgtC+0LPQvi?=
...

RFC2047: "The 'encoded-text' in an 'encoded-word' must be self-contained; 'encoded-text' MUST NOT be continued from one 'encoded-word' to another. This implies that the 'encoded-text' portion of a "B" 'encoded-word' will be a multiple of 4 characters long;".

As you see the text in first element has 63 characters. so it's an imap4d bug. However, we could probably implement a workaround.

[alecpl]

Fixed.