Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

attachment name is broken if it is in utf8 and longer than 24 characters #6048

Closed
z-eos opened this issue Nov 15, 2017 · 14 comments

Comments

Projects
None yet
2 participants
@z-eos
Copy link

commented Nov 15, 2017

attachment name is broken if it is utf8 and longer than 24 characters

resources

RC v.1.3.1, v.1.3.3
PHP v.5.6.30
imap4d (GNU Mailutils) 3.4

attached file name expected

абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx

attached file name provided by RC

Ð°Ð±Ð²Ð³Ð´ÐµÑ Ð¶Ð·Ð¸ÐºÐ»Ð¼Ð½Ð¾Ð¿Ñ Ñ Ñ Ñ Ñ Ñ Ñ ÑF#F'F+F_F3F7F;F_¹ ½ .docx

RC config

$config['mime_param_folding'] = 1;

original message fragment:

X-Envelope-Date: Tue Nov 14 09:16:53 2017
X-Envelope-Sender: jdoe@foo.bar

This is a multi-part message in MIME format.
--------------5BCCBC62D02AE2E4D591BEA1
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit



--------------5BCCBC62D02AE2E4D591BEA1
Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;
 name="=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE?=
 =?UTF-8?B?0YXRhtGH0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?="
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename*0*=utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7;
 filename*1*=%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82;
 filename*2*=%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C;
 filename*3*=%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78

UEsDBBQABgAIAAAAIQDfpNJsWgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIo
oAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

telnet session fragment

A0007 UID FETCH 6 (BODY.PEEK[2.MIME])
* 6 FETCH (UID 6 BODY[2.MIME] {565}
Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;
 name="=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE?=
 =?UTF-8?B?0YXRhtGH0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?="
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename*0*=utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7;
 filename*1*=%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82;
 filename*2*=%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C;
 filename*3*=%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78

)
A0007 OK UID FETCH Completed
A0008 UID FETCH 6 (BODY.PEEK[2])
* 6 FETCH (UID 6 BODY[2] {16196}
UEsDBBQABgAIAAAAIQDfpNJsWgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIo
oAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

RC debug fragment

Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: * 6 FETCH (UID 6 BODY[2.MIME] {565}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;^M  name="=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE?=^M  =?UTF-8?B?0YXRhtGH0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?="^M Content-Transfer-Encoding: base64^M Content-Disposition: attachment;^M  filename*0*=utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7;^M  filename*1*=%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82;^M  filename*2*=%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C;^M  filename*3*=%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78^M ^M
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: )
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: A0008 OK UID FETCH Completed
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] C: A0009 UID FETCH 6 (BODY.PEEK[2])
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: * 6 FETCH (UID 6 BODY[2] {16196}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: UEsDBBQABgAIAAAAIQDfpNJsWgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIo

miscelaneous

uudeview decodes correctly

uudeview 1510651013.M946286P86037Q0.relay.xx\:2,RS 

Loaded from 1510651013.M946286P86037Q0.relay.xx:2,RS: 'test' (test): абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx part 1   Base64
                                                                      
Found 'абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx' State 16 Base64 Parts 1 OK

  -rw-r--r-- абвгдеёжзиклмнопрстуф хцчшщъыьэюя.docx is OK   [d] (?=help)
@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 16, 2017

So, the message was created by Roundcube and you think it is malformed, or the message was created by some other client and Roundcube displays it wrong? Roundcube trusts BODYSTRUCTURE response, so check this in your debug log. I think it's more likely a bug on imap server side.

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 16, 2017

any MUA (thunderbird, K9, roundcube, default android MUA were tested) sending attachment as I described, causing RC to break attachment name

as for the server, it is not changing the message itself, but I'll check that as you suggest

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 16, 2017

the only difference I see is how RC receives name ... by chunks, while in telnet connection it comes as single piece \n delimited ...

telnet session

A0006 UID FETCH 6 (UID RFC822.SIZE FLAGS INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])
* 6 FETCH (UID 6 RFC822.SIZE 18020 FLAGS (\Seen) INTERNALDATE "14-Nov-2017 09:16:53 +0000" BODYSTRUCTURE (("text" "plain" ("charset" "utf-8" "format" "flowed") NIL NIL "7bit" 2 1 NIL NIL NIL)("application" "vnd.openxmlformats-officedocument.wordprocessingml.document" ("name" {118}
=?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?UTF-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=) NIL NIL "base64" 16196 NIL ("attachment" ("filename" {118}
=?utf-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?utf-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=)) NIL) "mixed" ("boundary" "------------5BCCBC62D02AE2E4D591BEA1") NIL "ru") BODY[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)] {348}
RETURN-PATH: <jdoe@foo.bar>
TO: jdoe@foo.bar
CC: jdoe2@foo2.bar
FROM: "jdoe@foo.bar" <jdoe@foo.bar>
SUBJECT: test
MESSAGE-ID: <8c1d39d7-5fef-8740-0607-b6f22fb3e742@xprmail.org>
DATE: Tue, 14 Nov 2017 11:16:17 +0200
CONTENT-TYPE: multipart/mixed;
 boundary="------------5BCCBC62D02AE2E4D591BEA1"

)
A0006 OK UID FETCH Completed

RC debug

Nov 15 23:14:57 rc roundcube: <qekqslgo> [2BAC] C: A0007 UID FETCH 6 (UID RFC822.SIZE FLAGS INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])
Nov 15 23:14:57 rc roundcube: <qekqslgo> [2BAC] S: * 6 FETCH (UID 6 RFC822.SIZE 18020 FLAGS (\Seen) INTERNALDATE "14-Nov-2017 09:16:53 +0000" BODYSTRUCTURE (("text" "plain" ("charset" "utf-8" "format" "flowed") NIL NIL "7bit" 2 1 NIL NIL NIL)("application" "vnd.openxmlformats-officedocument.wordprocessingml.document" ("name" {118}
Nov 15 23:14:57 rc roundcube: <qekqslgo> [2BAC] S: =?UTF-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?UTF-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: ) NIL NIL "base64" 16196 NIL ("attachment" ("filename" {118}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: =?utf-8?B?0LDQsdCy0LPQtNC10ZHQttC30LjQutC70LzQvdC+0L/RgNGB0YLRg9GE0YXRhtG?=  =?utf-8?B?H0YjRidGK0YvRjNGN0Y7Rjy5kb2N4?=
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: )) NIL) "mixed" ("boundary" "------------5BCCBC62D02AE2E4D591BEA1") NIL "ru") BODY[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)] {348}
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: RETURN-PATH: <jdoe@foo.bar>^M TO: jdoe@foo.bar^M CC: jdoe2@foo2.bar^M FROM: "jdoe@foo.bar" <jdoe@foo.bar>^M SUBJECT: test^M MESSAGE-ID: <8c1d39d7-5fef-8740-0607-b6f22fb3e742@xprmail.org>^M DATE: Tue, 14 Nov 2017 11:16:17 +0200^M CONTENT-TYPE: multipart/mixed;^M  boundary="------------5BCCBC62D02AE2E4D591BEA1"^M ^M
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: )
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] S: A0007 OK UID FETCH Completed
Nov 15 23:14:58 rc roundcube: <qekqslgo> [2BAC] C: A0008 UID FETCH 6 (BODY.PEEK[2.MIME])
@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 16, 2017

I'm not sure it's related, but my imap server returns also this:

("FILENAME*" {214}utf-8''%D0%B0%D0%B1%D0%B2%D0%B3%D0%B4%D0%B5%D1%91%D0%B6%D0%B7%D0%B8%D0%BA%D0%BB%D0%BC%D0%BD%D0%BE%D0%BF%D1%80%D1%81%D1%82%D1%83%D1%84%D1%85%D1%86%D1%87%D1%88%D1%89%D1%8A%D1%8B%D1%8C%D1%8D%D1%8E%D1%8F%2E%64%6F%63%78)

but if I comment out "filename" handling, it's still decoded correctly. So, I suppose the headers are encoded correctly. Could you provide a complete sample message?

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 16, 2017

@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 16, 2017

The sample works for me. So, more investigation needed. I'm using PHP 7.1, but I wouldn't say it's PHP's fault. Could you provide BODYSTRUCTURE response from your IMAP server for the same message?

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 16, 2017

here it is

A0005 OK [READ-WRITE] SELECT Completed
A0006 UID FETCH 9 (UID RFC822.SIZE FLAGS INTERNALDATE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])
* 9 FETCH (UID 9 RFC822.SIZE 348941 FLAGS (\Seen) INTERNALDATE "16-Nov-2017 07:58:31 +0000" BODYSTRUCTURE (("text" "plain" ("charset" "US-ASCII" "format" "flowed") NIL NIL "7bit" 37 3 NIL NIL NIL)("application" "pdf" ("name" {216}
=?UTF-8?B?Um91bmRjdWJlINC/0YDQuNC80LXRgCDQsdC40YLQvtCz0L4g0LjQvNC10L3QuCD?=  =?UTF-8?B?QstC70L7QttC10L3QuNGPINC00LvRjyDQv9GA0LjQvNC10YDQsCDRgtC+0LPQvi?=  =?UTF-8?B?DQutCw0Log0L7QvdC+INC10YHRgtGMINCx0YvRgtGMLnBkZg==?=) NIL NIL "base64" 346692 NIL ("attachment" ("filename" {216}
=?UTF-8?B?Um91bmRjdWJlINC/0YDQuNC80LXRgCDQsdC40YLQvtCz0L4g0LjQvNC10L3QuCD?=  =?UTF-8?B?QstC70L7QttC10L3QuNGPINC00LvRjyDQv9GA0LjQvNC10YDQsCDRgtC+0LPQvi?=  =?UTF-8?B?DQutCw0Log0L7QvdC+INC10YHRgtGMINCx0YvRgtGMLnBkZg==?= "size" "253352")) NIL) "mixed" ("boundary" "=_742fc19fa7c58c17da30ccc018454efc") NIL NIL) BODY[HEADER.FIELDS (DATE FROM TO SUBJECT CONTENT-TYPE CC REPLY-TO LIST-POST DISPOSITION-NOTIFICATION-TO X-PRIORITY IN-REPLY-TO BCC SENDER MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)] {308}
RETURN-PATH: <zeus@ibs.dn.ua>
CONTENT-TYPE: multipart/mixed;
 boundary="=_742fc19fa7c58c17da30ccc018454efc"
DATE: Thu, 16 Nov 2017 09:58:01 +0200
FROM: Zeus Panchenko <zeus@ibs.dn.ua>
TO: hostmaster@ibs.dn.ua
SUBJECT: attachment name broken
MESSAGE-ID: <7e9465080b75b81e2fd78de7ded5082a@ibs.dn.ua>

)
A0006 OK UID FETCH Completed
A0007 UID FETCH 9 (BODY.PEEK[2.MIME])
* 9 FETCH (UID 9 BODY[2.MIME] {1041}
Content-Transfer-Encoding: base64
Content-Type: application/pdf;
 name="=?UTF-8?Q?Roundcube_=D0=BF=D1=80=D0=B8=D0=BC=D0=B5=D1=80_=D0=B1=D0?=
 =?UTF-8?Q?=B8=D1=82=D0=BE=D0=B3=D0=BE_=D0=B8=D0=BC=D0=B5=D0=BD=D0=B8_=D0?=
 =?UTF-8?Q?=B2=D0=BB=D0=BE=D0=B6=D0=B5=D0=BD=D0=B8=D1=8F_=D0=B4=D0=BB=D1?=
 =?UTF-8?Q?=8F_=D0=BF=D1=80=D0=B8=D0=BC=D0=B5=D1=80=D0=B0_=D1=82=D0=BE=D0?=
 =?UTF-8?Q?=B3=D0=BE_=D0=BA=D0=B0=D0=BA_=D0=BE=D0=BD=D0=BE_=D0=B5=D1=81?=
 =?UTF-8?Q?=D1=82=D1=8C_=D0=B1=D1=8B=D1=82=D1=8C=2Epdf?="
Content-Disposition: attachment;
 filename*0*=UTF-8''Roundcube%20%D0%BF%D1%80%D0%B8%D0%BC%D0%B5%D1%80%20%D0;
 filename*1*=%B1%D0%B8%D1%82%D0%BE%D0%B3%D0%BE%20%D0%B8%D0%BC%D0%B5%D0%BD;
 filename*2*=%D0%B8%20%D0%B2%D0%BB%D0%BE%D0%B6%D0%B5%D0%BD%D0%B8%D1%8F%20;
 filename*3*=%D0%B4%D0%BB%D1%8F%20%D0%BF%D1%80%D0%B8%D0%BC%D0%B5%D1%80%D0;
 filename*4*=%B0%20%D1%82%D0%BE%D0%B3%D0%BE%20%D0%BA%D0%B0%D0%BA%20%D0%BE;
 filename*5*=%D0%BD%D0%BE%20%D0%B5%D1%81%D1%82%D1%8C%20%D0%B1%D1%8B%D1%82;
 filename*6*=%D1%8C.pdf;
 size=253352

)
A0007 OK UID FETCH Completed
@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 16, 2017

The second response is evidently malformed by the server.

So, what's going on here. Roundcube sees that there are filename* "fields" in the header. In such case it's fetching complete header and uses it to get the attachment name. Maybe we could detect that and fallback to the name specified in BODYSTRUCTURE response.

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 16, 2017

sorry much, I didn't catch in sight that but it was copy-paste error
I corrected the previous message
now all fields `filename' are where they have to be ...

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 16, 2017

was it helpful?

@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 17, 2017

Now I have no clue. While obviously your BODYSTRUCTURE response is different than mine, I'm talking about "filename" part, I'm not able to reproduce the issue even if I change the code to use your bodystructure.

What is your locale? What default character set (Preferences > Displaying Messages > Advanced Options)? Did you try different browser? Did you try different skin? Did you try with disabled all plugins? Do you have php-mbstring and/or php-iconv installed? Where exactly do you see the broken text, in message preview or when downloading the file?

@z-eos

This comment has been minimized.

Copy link
Author

commented Nov 17, 2017

  • locale is UTF8
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8
  • default charset is UTF8
  • browsers tested: FireFox (unix, windows), Chrome (windows)
  • Classic and Larry skins produce the same result
  • plugins disabled have no effect, though About still shows jquery and filesystem_attachments enabled even with $config['plugins'] = array(); in config
  • php-mbstring and php-iconv both are installed
  • I see the broken text, in both, message preview and file download dialogue
@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 17, 2017

I should have find this earlier. Anyway, there's a bug in handling names encoded this way:

=?UTF-8?B?Um91bmRjdWJlINC/0YDQuNC80LXRgCDQsdC40YLQvtCz0L4g0LjQvNC10L3QuCD?=
=?UTF-8?B?QstC70L7QttC10L3QuNGPINC00LvRjyDQv9GA0LjQvNC10YDQsCDRgtC+0LPQvi?=
...

RFC2047: "The 'encoded-text' in an 'encoded-word' must be self-contained; 'encoded-text' MUST NOT be continued from one 'encoded-word' to another. This implies that the 'encoded-text' portion of a "B" 'encoded-word' will be a multiple of 4 characters long;".

As you see the text in first element has 63 characters. so it's an imap4d bug. However, we could probably implement a workaround.

@roundcube roundcube deleted a comment from z-eos Nov 17, 2017

@roundcube roundcube deleted a comment from z-eos Nov 17, 2017

@alecpl alecpl added this to the 1.3.4 milestone Nov 17, 2017

alecpl added a commit that referenced this issue Nov 17, 2017

alecpl added a commit that referenced this issue Nov 17, 2017

@alecpl

This comment has been minimized.

Copy link
Member

commented Nov 17, 2017

Fixed.

@alecpl alecpl closed this Nov 17, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.