Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attachments ignored if BODYSTRUCTURE does not contain content disposition (Mercury/32 server) #7214

Open
informas opened this issue Feb 7, 2020 · 23 comments

Comments

@informas
Copy link

informas commented Feb 7, 2020

Hi,

Since updating Roundcube from version 1.3.6 to 1.4.2 I have problems with
some emails from GMAIL generated from a mobile phone with an attached image.

When viewing the email, Roundcube does not detect that it has an attachment,
but if I reply or forward that email it does.

I have detected that the problem appears when the attachment has a
"Content-ID:" valued header and is not used as an inline image.

For example, if the attached image is defined like this, Roundcube does not
detect the attachment:

Content-Type: image / jpeg; name = "image1.jpg"
Content-Disposition: attachment; filename = "image1.jpg"
Content-Transfer-Encoding: base64
Content-ID: <f_k6cdcld40>

On the other hand, if it is defined without Content-ID, Roundcube does
detect the attachment and preview the image:

Content-Type: image / jpeg; name = "image1.jpg"
Content-Disposition: attachment; filename = "image1.jpg"
Content-Transfer-Encoding: base64

A simplified version of the raw email to reproduce the problem is as follows:

MIME-Version: 1.0
From: FROMPERSON <fromabc@gmail.com>
Date: Fri, 7 Feb 2020 13:13:25 -0300
Subject: IM4 - with X-Attachment-Id AND with Content-ID
To: "DDESTINATION" <email@domain.com>
Content-Type: multipart/mixed; boundary="000000000000134e5e059dfeac92"
X-PMFLAGS: 570949760 0 5 YZD01IM4.CNM                       

--000000000000134e5e059dfeac92
Content-Type: multipart/alternative; boundary="000000000000134e5b059dfeac90"

--000000000000134e5b059dfeac90
Content-Type: text/plain; charset="UTF-8"

Text plain content

--000000000000134e5b059dfeac90
Content-Type: text/html; charset="UTF-8"

<div dir="ltr">HTML text content</div>

--000000000000134e5b059dfeac90--
--000000000000134e5e059dfeac92
Content-Type: image/jpeg; name="image1.jpg"
Content-Disposition: attachment; filename="image1.jpg"
Content-Transfer-Encoding: base64
Content-ID: <f_k6cdcld40>

/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQICAQECAQEBAgICAgICAgICAQICAgICAgICAgL/2wBDAQEBAQEBAQEBAQECAQEBAgICAgIC
AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/wgARCAAJAAgDASIA
AhEBAxEB/8QAFgABAQEAAAAAAAAAAAAAAAAAAAgJ/8QAFQEBAQAAAAAAAAAAAAAAAAAABwj/2gAM
AwEAAhADEAAAAdHUdGeYv//EABYQAAMAAAAAAAAAAAAAAAAAAAUQF//aAAgBAQABBQKhBl//xAAb
EQAABwEAAAAAAAAAAAAAAAAAAQMGFlOT0f/aAAgBAwEBPwGeOCxLIuj/xAAZEQABBQAAAAAAAAAA
AAAAAAAAAgUWUpL/2gAIAQIBAT8BjLXVez//xAAZEAABBQAAAAAAAAAAAAAAAAAEBRA0k9T/2gAI
AQEABj8CjKdIu1v/xAAUEAEAAAAAAAAAAAAAAAAAAAAQ/9oACAEBAAE/IR9//9oADAMBAAIAAwAA
ABDL/8QAFBEBAAAAAAAAAAAAAAAAAAAAAP/aAAgBAwEBPxBw/8QAFhEAAwAAAAAAAAAAAAAAAAAA
ANHx/9oACAECAQE/EL5H/8QAFBABAAAAAAAAAAAAAAAAAAAAEP/aAAgBAQABPxA9/wD/2Q==
--000000000000134e5e059dfeac92--

Has anyone had this problem? Could you solve it? Is it a program problem or
something that I have configured incorrectly?

Thanks in advance.
Jorge

@alecpl
Copy link
Member

alecpl commented Feb 8, 2020

Sounds like a duplicate of #7122.

@alecpl alecpl closed this as completed Feb 8, 2020
@alecpl
Copy link
Member

alecpl commented Feb 8, 2020

On the other hand, it works for me in git-master. Maybe fixed with #7117.

@informas
Copy link
Author

informas commented Feb 8, 2020

Hi, thank you very much for your answer.

While this inconvenience may be similar to that reported in #7122 it is somewhat different.

In #7122, I understand, the inconvenience occurs when responding to emails and attaching files.

In my case, the attachments are not shown in the email view either as a preview or as a new window.
Not only that the content of the attachment is not shown, if applicable, but the list of attachments is also not shown.

But, the particular thing is that if I have forwarded that email (of which I don't see the list of attachments), in the compose window appears the list with the attachment.

That as it with RC version 1.4.2 downloaded on 01/07/2020.

Today, after your response referring to #7117, I downloaded the updated version (as of today, 02/08/2020) and not only that the problem reported is not solved but that now the list of attachments is not shown when trying to resend the email.

Jorge

@alecpl
Copy link
Member

alecpl commented Feb 9, 2020

As I've said, it works for me. What exactly did you download? git-master?

@informas
Copy link
Author

informas commented Feb 9, 2020

Hi,
I have downloaded it from https://github.com/roundcube/roundcubemail/archive/master.zip and unzip them over the previous instalation.

I downloaded the correct version?

But I preserves the my own config and plugins.

Could there be any conflict with some plugin?

TIA

Jorge

@informas
Copy link
Author

Hi,
Please excuse me for returning to this closed topic.
Unfortunately I still have the same problem, even after updating to the latest version of Roundcube, 1.4.4.

I will try to better describe the problem.

For some emails that have images sent as attachments when they are displayed both in the preview or when they are opened in a new window, the attached images are not displayed.

Most of those emails that present the described problem are sent from cell phones attaching images of photographs taken with this device.

I have found as a common condition to all these cases that the attachment data in the email contains the field "Content-ID", for example:

Content-Type: image / jpeg; name = "image1.jpg"
Content-Disposition: attachment; filename = "image1.jpg"
Content-Transfer-Encoding: base64
Content-ID: <f_k6cdcld40>

Under this condition the email preview appears like this (as if it had no attachment):
with-content-id

But if instead of that email in the definition of the attachment did not have the Content-ID field, both the preview and the normal view of the email show the attachment (if it is an image) and the information about the attached files.

As an example, I have modified the email to alter the definition of the attachment, as follows (I have removed the Content-ID field):

Content-Type: image / jpeg; name = "image1.jpg"
Content-Disposition: attachment; filename = "image1.jpg"
Content-Transfer-Encoding: base64

And in that case the email is displayed as expected with the attached image and the corresponding information (see following image):
without-content-id

Please, based on this more detailed and illustrated description, could you review the problem?

As additional information I comment that I have done these tests deactivating all the plugins.

Is there any additional information I can provide?

Thanks in advance,

@alecpl
Copy link
Member

alecpl commented May 13, 2020

Full message source, please.

@alecpl alecpl reopened this May 13, 2020
@informas
Copy link
Author

Hi Alec,

With Content-ID version:

MIME-Version: 1.0
From: FROMPERSON <fromabc@gmail.com>
Date: Fri, 8 May 2020 13:20:21 -0300
Subject: With X-Attachment-Id AND with Content-ID
To: "DESTINATION" <email@domain.com>
Content-Type: multipart/mixed; boundary="000000000000134e5e059dfeac92"
X-PMFLAGS: 570949760 0 5 RCCC7214.CNM                       

--000000000000134e5e059dfeac92
Content-Type: multipart/alternative; boundary="000000000000134e5b059dfeac90"

--000000000000134e5b059dfeac90
Content-Type: text/plain; charset="UTF-8"

Text plain content

--000000000000134e5b059dfeac90
Content-Type: text/html; charset="UTF-8"

<div dir="ltr">HTML text content</div>

--000000000000134e5b059dfeac90--
--000000000000134e5e059dfeac92
Content-Type: image/jpeg; name="image1.jpg"
Content-Disposition: attachment; filename="image1.jpg"
Content-Transfer-Encoding: base64
Content-ID: <f_k6cdcld40>

/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQICAQECAQEBAgICAgICAgICAQICAgICAgICAgL/2wBDAQEBAQEBAQEBAQECAQEBAgICAgIC
AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/wgARCAAJAAgDASIA
AhEBAxEB/8QAFgABAQEAAAAAAAAAAAAAAAAAAAgJ/8QAFQEBAQAAAAAAAAAAAAAAAAAABwj/2gAM
AwEAAhADEAAAAdHUdGeYv//EABYQAAMAAAAAAAAAAAAAAAAAAAUQF//aAAgBAQABBQKhBl//xAAb
EQAABwEAAAAAAAAAAAAAAAAAAQMGFlOT0f/aAAgBAwEBPwGeOCxLIuj/xAAZEQABBQAAAAAAAAAA
AAAAAAAAAgUWUpL/2gAIAQIBAT8BjLXVez//xAAZEAABBQAAAAAAAAAAAAAAAAAEBRA0k9T/2gAI
AQEABj8CjKdIu1v/xAAUEAEAAAAAAAAAAAAAAAAAAAAQ/9oACAEBAAE/IR9//9oADAMBAAIAAwAA
ABDL/8QAFBEBAAAAAAAAAAAAAAAAAAAAAP/aAAgBAwEBPxBw/8QAFhEAAwAAAAAAAAAAAAAAAAAA
ANHx/9oACAECAQE/EL5H/8QAFBABAAAAAAAAAAAAAAAAAAAAEP/aAAgBAQABPxA9/wD/2Q==
--000000000000134e5e059dfeac92--

WITHOUT Content-ID version:

MIME-Version: 1.0
From: FROMPERSON <fromabc@gmail.com>
Date: Fri, 8 May 2020 13:14:25 -0300
Subject: With X-Attachment-Id AND WITHOUT Content-ID
To: "DESTINATION" <email@domain.com>
Content-Type: multipart/mixed; boundary="000000000000134e5e059dfeac92"
X-PMFLAGS: 570949760 0 5 RCSC7214.CNM                       

--000000000000134e5e059dfeac92
Content-Type: multipart/alternative; boundary="000000000000134e5b059dfeac90"

--000000000000134e5b059dfeac90
Content-Type: text/plain; charset="UTF-8"

Text plain content

--000000000000134e5b059dfeac90
Content-Type: text/html; charset="UTF-8"

<div dir="ltr">HTML text content</div>

--000000000000134e5b059dfeac90--
--000000000000134e5e059dfeac92
Content-Type: image/jpeg; name="image1.jpg"
Content-Disposition: attachment; filename="image1.jpg"
Content-Transfer-Encoding: base64

/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQICAQECAQEBAgICAgICAgICAQICAgICAgICAgL/2wBDAQEBAQEBAQEBAQECAQEBAgICAgIC
AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/wgARCAAJAAgDASIA
AhEBAxEB/8QAFgABAQEAAAAAAAAAAAAAAAAAAAgJ/8QAFQEBAQAAAAAAAAAAAAAAAAAABwj/2gAM
AwEAAhADEAAAAdHUdGeYv//EABYQAAMAAAAAAAAAAAAAAAAAAAUQF//aAAgBAQABBQKhBl//xAAb
EQAABwEAAAAAAAAAAAAAAAAAAQMGFlOT0f/aAAgBAwEBPwGeOCxLIuj/xAAZEQABBQAAAAAAAAAA
AAAAAAAAAgUWUpL/2gAIAQIBAT8BjLXVez//xAAZEAABBQAAAAAAAAAAAAAAAAAEBRA0k9T/2gAI
AQEABj8CjKdIu1v/xAAUEAEAAAAAAAAAAAAAAAAAAAAQ/9oACAEBAAE/IR9//9oADAMBAAIAAwAA
ABDL/8QAFBEBAAAAAAAAAAAAAAAAAAAAAP/aAAgBAwEBPxBw/8QAFhEAAwAAAAAAAAAAAAAAAAAA
ANHx/9oACAECAQE/EL5H/8QAFBABAAAAAAAAAAAAAAAAAAAAEP/aAAgBAQABPxA9/wD/2Q==
--000000000000134e5e059dfeac92--

As you can see, both emails have the same content, and in what they differ it is only in the definition block of the attachment, that one (the first, which malfunctions) has the Content-ID field while the other (the last, that works well) doesn't have it.

Attached both files.

Thanks in advance.

with-contentid.txt
without-contentid.txt

@alecpl
Copy link
Member

alecpl commented May 13, 2020

It works for me. What IMAP server are you using? Enable imap_debug and check in the log what the server returns in BODYSTRUCTURE response.

@informas
Copy link
Author

Hi,
I am using Mercury/32 mail server version v4.80.145 under Windows 10.

Attached you will find the requested log.

I have extracted the specific part that corresponds to my work session in which, once imap_debug was enabled, I first checked the email "with problems" (the one that does not show the image) and then the one that normally shows it.

The attached file contains all the complete sequence I debugged.

Do you need any additional information?

Thanks in advance.
my-imap-session.txt

@alecpl
Copy link
Member

alecpl commented May 14, 2020

So, we're talking about this: BODYSTRUCTURE ((("text" "plain" ("CHARSET" "UTF-8") NIL NIL "7BIT" 21 2 NIL NIL NIL)("text" "html" ("CHARSET" "UTF-8") NIL NIL "7BIT" 41 2 NIL NIL NIL) "alternative" ("BOUNDARY" "000000000000134e5b059dfeac90") NIL NIL NIL)("image" "jpeg" ("NAME" "image1.jpg") "f_k6cdcld40" NIL "BASE64" 775 NIL NIL NIL) "mixed" ("BOUNDARY" "000000000000134e5e059dfeac92") NIL NIL NIL).

Your server does not return Content-Disposition definition in the structure. The image part should look like this: ("IMAGE" "JPEG" ("NAME" "image1.jpg") "<f_k6cdcld40>" NIL "BASE64" 774 NIL ("ATTACHMENT" ("FILENAME" "image1.jpg")) NIL NIL).

This is the main problem here. I have an idea how we could workaround that. I'll work on this.

@alecpl alecpl changed the title Problems previewing some attached imagesin Roundcube Version 1.4.2 Attachments ignored if BODYSTRUCTURE does not contain content disposition (Mercury/32 server) May 14, 2020
@alecpl alecpl added this to the later milestone May 14, 2020
@informas
Copy link
Author

First of all, thank you very much for your predisposition.

If the additional information can be of any use, I tell you that I have tried the same situation with Roundcube version 1.3.6 and in that version the problem appears in part. In that version I have implemented the Larry skin.

Roundcube 1.3.6, in both cases detects the attachment, although the attachment information contains Content-ID is not shown in the preview. (See images)

Roundcube v: 1.3.6 - Skin: Larry - Attachment with Content-ID
RCv136-Larry-ContentID

Roundcube v: 1.3.6 - Skin: Larry - Attachment WITHOUT Content-ID
RCv136-Larry-woContentID

I also tested version 1.4.4, with the Larry skin and the behavior is identical to that described for the Elastic skin. (See image) That is, when the attachment information contains the Content-ID field, it does not show the list of attachments or the attached images in the preview.

Roundcube v: 1.4.4 - Skin: Larry - Attachment with Content-ID
RCv144-Larry-ContentID

Roundcube v: 1.4.4 - Skin: Larry - Attachment WITHOUT Content-ID
RCv144-Larry-woContentID

I hope this information helps to find a solution to the problem.

If I wanted to review the code, to try to find a solution, in which modules of the Roundcube code should I look for the treatment of this topic?

Thanks in advance.

@alecpl
Copy link
Member

alecpl commented May 14, 2020

This has nothing to do with skins. The fix would have to be in rcube_imap::structure_part() or rcube_message::parse_structure() or in program/steps/mail/show.inc.

I investigated a bit and it does not look that easy to fix without breaking other things. In a way it is related to #5051.

@informas
Copy link
Author

informas commented May 14, 2020

I get it. I will wait for you to find some solution to the problem.
For the moment, thank you very much.

@informas
Copy link
Author

I would like to clarify one point.
Although the mail server I use is Mercury / 32, in my humble opinion, I do not think it has to do with the inconvenience since the questioned email does not originate from this server but rather this is the one that receives the email.
In real cases (since the sample email I have generated is a simple one to replicate the situation and test) it is an email that is generated from the email client of some cell phones, the attached image being one obtained with the same phone and then shared from the mail.
In most cases of the cell phones that originate these emails, the sending email account is a Gmail domain account and consequently the originating server is GMail, my Mercury / 32 server only acts as a relay.
Does this comment provide any additional elements?

@alecpl
Copy link
Member

alecpl commented May 15, 2020

Not really. The issue here is that your specific server ignores the Content-Disposition header (and do not pass it to Roundcube), which is one of the ways to decide whether an email part is attachment or inline. Without this information we have to make assumptions. And indeed we treat the image as inline in this case because of Content-Id existence.

I'm not saying we cannot improve the logic on Roundcube side (it's not perfect and some tickets for this already exist). But when I investigated the code I found contradictory code branches that try to workaround email client bugs, like for example sending inline attachments inside mixed parts. So, if we try to fix one thing another might stop working.

So, we have to be careful and find a good compromise or a proper solution if at all possible (which might involve a bigger refactoring). The ticket I pointed on before is exactly about one of the possible solutions also for this case. Treat all attachments (images) that aren't referred in the HTML body as non-inline parts.

@informas
Copy link
Author

Hi Alec,

According to your answer, I was reviewing the code of the rcube_message module and I was able to adapt it to solve the consulted problem, at least temporarily until there is officially a definitive solution.

I do not know much about the complete system or its characteristics, so it is very likely that the solution adopted is not the best or technically correct, so if it is possible that you or someone else has a better approach, please let me know.

For the moment and as far as I have been able to test, with the modification made I solve the problem and do not introduce any side effects (until they appear!;))

The modification made consists of the following:

In rcube_message::parse_structure(), between lines 845 and 846 add:

L844: if ($mail_part->headers['content-id']) {
L845:     $mail_part->content_id = preg_replace(array('/^</', '/>$/'), '', $mail_part->headers['content-id']);

+         if (     empty($mail_part->headers['content-type'])
+              and empty($mail_part->headers['content-disposition'])
+              and $this->headers->ctype == 'multipart/mixed'
+            ) 
+         {
+            $mail_part->headers['content-type'] = "{$mail_part->mimetype}; name=\"{$mail_part->filename}\"";
+            $mail_part->headers['content-disposition'] = "attachment;  name=\"{$mail_part->filename}\"";
+            $mail_part->disposition = '';
+         }

L846: }
L847:
L848: if ($mail_part->headers['content-location']) {

My explanation:
If the header of the attachment 'content-id' has value and 'content-type' and 'content-disposition' are not valued and also the content-type of the email is 'multipart/mixed' (all conditions that are verified for emails that present the consulted problem), then assign value to 'content-type' and 'content-disposition' and to $mail_part->disposition assigns ''.
In this way I manage to equate the behavior to the cases that are shown correctly.

I will appreciate any suggestion or comment that anyone can make.

@alecpl
Copy link
Member

alecpl commented May 19, 2020

I was thinking more in the line of:

--- a/program/lib/Roundcube/rcube_imap.php
+++ b/program/lib/Roundcube/rcube_imap.php
@@ -2189,10 +2189,6 @@ class rcube_imap extends rcube_storage
         // get part ID
         if (!empty($part[3])) {
             $struct->content_id = $struct->headers['content-id'] = trim($part[3]);
-
-            if (empty($struct->disposition)) {
-                $struct->disposition = 'inline';
-            }
         }
 
         // fetch message headers if message/rfc822 or named part (could contain Content-Location header)

but this of course has implications and requires more consideration.

@KylePinkley
Copy link

KylePinkley commented Jul 27, 2020

I was thinking more in the line of:

--- a/program/lib/Roundcube/rcube_imap.php
+++ b/program/lib/Roundcube/rcube_imap.php
@@ -2189,10 +2189,6 @@ class rcube_imap extends rcube_storage
         // get part ID
         if (!empty($part[3])) {
             $struct->content_id = $struct->headers['content-id'] = trim($part[3]);
-
-            if (empty($struct->disposition)) {
-                $struct->disposition = 'inline';
-            }
         }
 
         // fetch message headers if message/rfc822 or named part (could contain Content-Location header)

but this of course has implications and requires more consideration.

I have a customer that can't view PDF inline attachments in Roundcube (1.4.6) in an HTML message, that originates from a Microsoft SMTP Server mail server. We can't provide the full E-Mail headers unfortunately or a stripped version of it due to privacy per the customer.

I did try the patch mentioned above temporarily just to see if that would fix it and it does. The PDF and other attachments (JPG) does show. Since it is not a official patch, I reverted.

@informas
Copy link
Author

I have tried the modification proposed by Alec in 05-19 and that you also tried, as you comment.

It works well in the original case that I presented, that is, when the attachment is an image (for example a JPEG)

But ..., with this modification, emails that have embedded images stop showing correctly. For example when the html code is as:

<IMG SRC = "cid: cid-image-id">

In those cases, a well-formed email should look, for example, like this:
imagen

But, with the modification proposed by Alec it looks like this:
First (in the presentation header) shows a list with all the attached images. It shouldn't show it.
The body of the message shows it with the images correctly embedded in the HTML, as it should be.
imagen

But, also at the end of the email, it showes all the images that were already shown in the body of the email, one after another.
imagen

My conclusion: for my specific problem I prefer the solution initially adopted, i.e. modifying rcube_message.php

@geby
Copy link

geby commented Jan 11, 2021

I have this problem on 1.4.10. Is it still this old problem, or new regression?

@gmariani
Copy link

I have the same problem on 1.6.0 where an email with a JPEG attachment has a Content-ID causing it not to display or be downloadable.

------=_Part_3_1489400240.1698411486879
Content-Type: image/jpeg; name=IMG_0937.jpg
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename=IMG_0937.jpg
Content-ID: <1>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants