Attachments not visible in mail clients when additional inline attachments present #2686

Open
icanhasserver opened this Issue Aug 25, 2011 · 50 comments

Projects

None yet
@icanhasserver

When assembling an email with mixed inline / normal attachments, only the inline attachments (i.e. images) are shown. Some mail clients don't detect the attached files, the most prominent being Thunderbird and Outlook.

Code used with ActionMailer 3.0.10:

class MultipartTest < ActionMailer::Base
  def test_email(recipient)
    attachments['file1.pdf'] = File.read('/somewhere/file1.pdf')
    attachments['file2.pdf'] = File.read('/somewhere/file2.pdf')
    attachments.inline['image1.gif'] = File.read('/somewhere/image1.gif')
    mail(
      :to => recipient,
      :subject => 'Multipart test'
    )
  end
end

The generated email comes as follows:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

While ActionMailer::Base::set_content_type() chooses multipart/related as soon as it detects at least one inline attachment, mail clients always wrap the attached files (disposition: attachment) in an additional multipart/mixed layer, conforming to RFC 2046, Section 5.1.3.

When leaving out the inline attachement, the MIME type generated by ActionMailer is correct (multipart/mixed) and the attachments are visible.

Here are some MIME layouts, as generated by mail clients:

multipart/mixed
  multipart/alternative
    text/plain
    multipart/related
      text/html
      attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

or:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)
@noniq
Contributor
noniq commented Sep 30, 2011

I can confirm this problem. Our workaround for now is to avoid inline attachments at all if there are any normal attachments.

@epoch
epoch commented Nov 3, 2011

I can confirm this as well.

@tuggdev
tuggdev commented Nov 10, 2011

Confirm, Outlook can see attachments only after removing inline attachments.

@bradhodges

Yep, same here, on my iMac, the normal attachments show in the attachments pulldown, but the email itself has no indication that there are normal attachments.

Maybe a blessing? The inline attachment is just a little company logo in the emails 'signature' , something my users like, but I've always found annoying!

@isaacsanders
Contributor

Is this still an issue? cc @spastorino @tenderlove

@icanhasserver

I repeated the tests using ActionMailer 3.2.3 and I see no difference in the generated MIME structure: there's no "multipart/mixed".
So: this issue hasn't been solved, as far as I can see.

@dmkl
dmkl commented Sep 12, 2012

I confirm this issue as well.

@steveklabnik
Member

@jonleighton is your work on basecamp/mail_view#17 useful here, too? Any plans on fixing this in Rails as well as that app?

@jonleighton
Member

@steveklabnik I want to fix this at some point; it might become a necessity through my work actually. No ETA though.

@steveklabnik
Member

Fair enough! Thanks.

@lathiat
lathiat commented Oct 11, 2012

This hit me too, in particular this is blocking sending out e-mails with Passbook attachments (while also including inline files) because the iOS Mail client refuses to see the passkit file attached due to this issue. The Mac Mail.app client however shows it fine.

@jonleighton
Member

I've come up with a workaround for this bug. Add this method to your mailer:

  # Workaround for https://github.com/rails/rails/issues/2686
  def fix_mixed_attachments
    mail = Mail.new
    mail.delivery_method delivery_methods[delivery_method.to_sym], public_send("#{delivery_method}_settings")

    related = Mail::Part.new
    related.content_type = @_message.content_type
    @_message.parts.select { |p| !p.attachment? || p.inline? }.each { |p| related.add_part(p) }
    mail.add_part related

    mail.header       = @_message.header.to_s
    mail.content_type = nil
    @_message.parts.select { |p| p.attachment? && !p.inline? }.each { |p| mail.add_part(p) }

    @_message = mail
  end

Then call it at the end of your action method:

  def notification(...)
    attachments['omg.pdf'] = ...
    attachments.inline['wtf.png'] = ...

    mail(...)
    fix_mixed_attachments
  end

YMMV etc, but it works for me.

@aaronjensen
Contributor

We're seeing that when attaching a text/csv when we have an html and text view a multipart/alternative wrapper (which includes another multipart/alternative and the attachment itself). We're not including any inline attachments. The outer multipart/alternative wrapper isn't understood by many mail clients. Is this a different issue or related?

@parndt
Contributor
parndt commented May 4, 2013

So based on the original bug and the examples of real mail clients and the workaround posted by @jonleighton it seems like the solution would be to just move attachment (disposition: inline, image1) (for example) underneath a multipart/related heading?

@hlascelles

We see this issue with Outlook 365 only. The same emails to gmail appear to be displayed fine.

Either way, the workaround from @jonleighton seems to work for us. Thanks!

@yyyc514
Contributor
yyyc514 commented Jun 27, 2013

Ping - any progress on getting a fix for this into Rails proper?

@yyyc514
Contributor
yyyc514 commented Jun 28, 2013

Improvement to the workaround earlier:

  • Aborts if there are no regular attachments, leaving the original message as it was
  • Calls a higher-level AM method (wrap_delivery_behavior!) that preserves settings like :perform_deliveries and raise_delivery_errors
  def fix_mixed_attachments
    # do nothing if we have no actual attachments
    return if @_message.parts.select { |p| p.attachment? && !p.inline? }.none?

    mail = Mail.new

    related = Mail::Part.new
    related.content_type = @_message.content_type
    @_message.parts.select { |p| !p.attachment? || p.inline? }.each { |p| related.add_part(p) }
    mail.add_part related

    mail.header       = @_message.header.to_s
    mail.content_type = nil
    @_message.parts.select { |p| p.attachment? && !p.inline? }.each { |p| mail.add_part(p) }

    @_message = mail
    wrap_delivery_behavior!(delivery_method.to_sym)
  end

Not using this in production yet but made the improvements during my testing in development.

@steveklabnik
Member

If you find something that works well, you should submit a pull request!

@kitebuggy

yyyc514's workaround seems to have fixed it for me. Thank you.

P.S. Would be nice to see this merged into the main code though...

@magpieuk

yyyc514's workaround also worked for me. For some reason Johns workaround worked in most cases but the attachment vanished in Outlook 2010 but was visible in Outlook 2013. When the missing email was forwarded back to me Outlook 2013 could now see the attachment. weird!

@d-ark
d-ark commented Apr 7, 2014

I used to have the same problem in my app. I've solved it changing message structure to this:

multipart/related
  multipart/related
    multipart/alternative
      text/plain
      text/html
    image/jpeg   #inline
    image/jpeg   #inline
    ...          #other inline attachments  
  multipart/related
    application/pdf #non-inline
    text/plain      #non-inline
    ...             #other non-inline attachments

Maybe it seems strange, that all non-inline attachments are wrapped by extra 'related'. I've tested this structure on different mail clients - it works fine. If we don't wrap them, and we have plain/text attachment in our message, some mail clients (gmail as well) parse this attachment as mail text part (it's displayed in previews).

I'd like to fix this issue, but i can't find code where the mail structure is formed... Maybe this is issue of mail gem?

cc / @steveklabnik @jonleighton

@yyyc514
Contributor
yyyc514 commented Apr 7, 2014

Building the structure is part of ActionMailer, you'll find the code there. Mail just provides the building blocks, nothing more.

@NikoRoberts

So should a PR be made to ActionMailer like:
http://github.com/rails/rails/commit/311d99eef01c268cedc6e9b3bdb9abc2ba5c6bfa
Or should a PR be made into Mail?

@rafaelfranca
Member

I believe we should fix Mail gem if it is broken. Who can investigate this issue and work on it?

@yyyc514
Contributor
yyyc514 commented Aug 1, 2014

This isn't Mail's problem (as the division of labor currently stands). Mail is the low-level email library and Rails is responsible for building the MIME envelopes for multi-part emails. Someone on the Rails team needs to care about this issue - which doesn't seem to be the case.

@rafaelfranca
Member

The Rails team don't have time to fix all the issues as I said before, who want to investigate and provide a pull request?

@yyyc514
Contributor
yyyc514 commented Aug 1, 2014

I'd be happy to, but I don't really know what the actual CORRECT solution is. You see my code above... I could merge it into AM proper... but is this the actual correct solution to the problem? I guess I was hoping someone would come out of the woodwork with an authoritative fix, vs "this works for me".

I'd worry without some official reference that the Core team wouldn't be as likely to merge the pull request since if they had run into this personally you'd think they'd have fixed it already. So those are my fears.

@yyyc514
Contributor
yyyc514 commented Aug 1, 2014

So it's the "investigate" part I guess I'd be hung up on. :)

@rafaelfranca
Member

I'll suggest to you open a pull request with your current fix. As you said we don't have this problem as would be way easier if we were discussing about a failing test to understand and some work in progress patch.

Could you do it?

@rafaelfranca rafaelfranca reopened this Aug 1, 2014
@yyyc514
Contributor
yyyc514 commented Aug 1, 2014

No idea how to write a failing test. Some email clients just don't like some aspect of how you currently build the MIME packets... not even sure if it's right or wrong we're talking about. Could be those clients (even if popular) are the problem... so this isn't really something that has a failing test. It's just that doing it an alternative way seems to work better across multiple clients.

This thread already should contain enough information to discuss the issue. All you could do is write a test that looked for the MIME structure recommended here and then failed... but that's no good unless people understand the issue and agree that the MIME structure above is the right way to go.

@yyyc514
Contributor
yyyc514 commented Aug 1, 2014

Can we agree which one of these is correct?

multipart/mixed
  multipart/alternative
    text/plain
    multipart/related
      text/html
      attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)
multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)
@matthewd
Member
matthewd commented Aug 1, 2014

Isn't that going to depend on the author's intent? IMO, the former is generally more "correct": the (inline) images "belong" to the HTML part, and should be ignored by anything that chooses to use the Plain part.

But the most realistic definition of correct will be in which elicits a more suitable behaviour from more clients, if there's any difference.

@ajb ajb referenced this issue in afeld/tricle Sep 2, 2014
Closed

include sparkline for each metric #29

@swelther

thx yyyc514 for the fix, it works for me.

But it omits the BCC from the header. I inserted this

mail.bcc = @_message.header[:bcc].value

as a workaround. Maybe there is a better way.

Maybe someone else stumbles over this too.

@yyyc514
Contributor
yyyc514 commented Nov 14, 2014

@swelther Are you sure you're using my code? There is a line that copies over all the headers whole sale... if it didn't work a lot more than the BCC would be broken. Have you tried to track down the issue any further? Honestly not sure why BCC would be some sort of edge case - though I haven't dug in further either.

mail.header       = @_message.header.to_s
@swelther

@yyyc514 Yes, it's your code. I was wondering too, maybe it is a documented behavior in newer Rails versions?

def fix_mixed_attachments
  # do nothing if we have no actual attachments
  return if @_message.parts.select { |p| p.attachment? && !p.inline? }.none?

  mail = Mail.new

  related = Mail::Part.new
  related.content_type = @_message.content_type
  @_message.parts.select { |p| !p.attachment? || p.inline? }.each { |p| related.add_part(p) }
  mail.add_part related

  mail.header = @_message.header.to_s
  mail.content_type = nil
  @_message.parts.select { |p| p.attachment? && !p.inline? }.each { |p| mail.add_part(p) }

  @_message = mail
  wrap_delivery_behavior!(delivery_method.to_sym)
end

this is what I get from @_message.header.to_s in my spec:

> @_message.header.to_s
=> "From: service@mail.com\r\nTo: fred@feuerstein.com\r\nSubject: blah\r\nMime-Version: 1.0\r\nContent-Type: multipart/related;\r\n boundary=\"--==_mimepart_5465f952f0ba1_7dde50133838765\";\r\n charset=UTF-8\r\n"

no bcc. But it is set in the header:

> @_message.header[:bcc].value
=> ["swelther@mail.com"]

Maybe Rails 4.0.10 (or active mailer or whatever) acts in this case a bit different than previous versions? IIRC I tried this on a server and got the same behavior as in my spec.

@yyyc514
Contributor
yyyc514 commented Nov 14, 2014

Oh, that seems intentional then. I'll look at it later. Perhaps you can paste another full code excerpt here and add the BCC fix... I'm guessing it's something intentional relating to the strange nature of BCC.

@swelther

sure, no big deal, it works for me 😀

so if anyone stumbles over not-copied BCC values use this version:

  def fix_mixed_attachments
    # do nothing if we have no actual attachments
    return if @_message.parts.select { |p| p.attachment? && !p.inline? }.none?

    mail = Mail.new

    related = Mail::Part.new
    related.content_type = @_message.content_type
    @_message.parts.select { |p| !p.attachment? || p.inline? }.each { |p| related.add_part(p) }
    mail.add_part related

    mail.header       = @_message.header.to_s
    mail.bcc          = @_message.header[:bcc].value # copy bcc manually because it is omitted in header.to_s
    mail.content_type = nil
    @_message.parts.select { |p| p.attachment? && !p.inline? }.each { |p| mail.add_part(p) }

    @_message = mail
    wrap_delivery_behavior!(delivery_method.to_sym)
  end
@yyyc514
Contributor
yyyc514 commented Nov 30, 2014

From Mail's bcc_field.rb line 24:

#  mail[:bcc].encoded   #=> ''      # Bcc field does not get output into an email

So BCC is a magic field that is never output (why to_s fails) but rather is (I'd imagine) used internally by Mail (or other gems) during the delivery process. So that's why it's necessary to copy it in such fashion. There may be a simpler way to just copy the whole header object over as-is (or rather clone it), but I didn't spend any time looking into that. I just wanted to answer the question of "why".

@swelther
swelther commented Dec 2, 2014

thx for clarifying @yyyc514. Sounds reasonable. Maybe someone gets bored and develops a simpler version 😀

@rails-bot rails-bot added the stale label Aug 25, 2015
@rails-bot

This issue has been automatically marked as stale because it has not been commented on for at least
three months.

The resources of the Rails team are limited, and so we are asking for your help.

If you can still reproduce this error on the 4-2-stable, 4-1-stable branches or on master,
please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions.

@javinto
javinto commented Oct 15, 2015

I can still reproduce this issue with Rails 4.2.4

The same e-mail sent to Apple Mail is correctly readable, but sent to a iPhone the inline logo is readable, but the attachments are not.

There has already been some discussion and solutions in the past: http://stackoverflow.com/questions/1118592/problem-sending-multipart-mail-using-actionmailer

But the provided solutions there do not work with Rails 4.x

@dmitry
Contributor
dmitry commented Oct 15, 2015

@javinto any more details?

@javinto
javinto commented Oct 15, 2015

@dmitry no my use case basically is similar to that of icanhasserver. My problem occurs with Rails 4.1 but I can still repeat it with Rails 4.2.4 on my IOS 9.02 device.

@pixeltrix pixeltrix self-assigned this Oct 15, 2015
@pixeltrix
Member

@javinto @dmitry I have a failing test now - hopefully the fix shouldn't be too painful

@dannolan

I just got burned by this this morning @pixeltrix, came looking and 10 hours prior someone's found it also. Let me know if I can do anything to help on this issue.

@pixeltrix
Member

@dannolan it'd be helpful if you can cast your eye over the possible structures below and point out any flaws:

# single, none
text/html

# single, inline
multipart/related
  text/html
  image/png

# single, attachment
multipart/mixed
  text/html
  application/pdf

# single, mixed
multipart/mixed
  multipart/related
    text/html
    image/png
  application/pdf

# multiple, none
multipart/alternative
  text/plain
  text/html

# multiple, inline
multipart/related
  multipart/alternative
    text/plain
    text/html
  image/png

# multiple, attachment
multipart/mixed
  multipart/alternative
    text/plain
    text/html
  application/pdf

# multiple, mixed
multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    image/png
  application/pdf
@javinto
javinto commented Oct 16, 2015

I'm sorry. After a simple test from Apple Mail I cannot recognize the structures below. I'm not into mail structures.
How can I recognize the indented structures from the Source view? There is no multipart/related used by Apple mail in my tests.
Does your list conform to the official specs?

Op 16 okt. 2015, om 12:04 heeft Andrew White notifications@github.com het volgende geschreven:

@dannolan https://github.com/dannolan it'd be helpful if you can cast your eye over the possible structures below and point out any flaws:

single, none

text/html

single, inline

multipart/related
text/html
image/png

single, attachment

multipart/mixed
text/html
application/pdf

single, mixed

multipart/mixed
multipart/related
text/html
image/png
application/pdf

multiple, none

multipart/alternative
text/plain
text/html

multiple, inline

multipart/related
multipart/alternative
text/plain
text/html
image/png

multiple, attachment

multipart/mixed
multipart/alternative
text/plain
text/html
application/pdf

multiple, mixed

multipart/mixed
multipart/related
multipart/alternative
text/plain
text/html
image/png
application/pdf

Reply to this email directly or view it on GitHub #2686 (comment).

@pixeltrix
Member

@javinto I'm using the pattern in the original report by @icanhasserver - top level is message content type and then each indent is a different nested part level and each line is a part and the text is the content type.

@psifertex psifertex referenced this issue in nodemailer/nodemailer Jul 29, 2016
Closed

nodemailer presenting attachments as alternatives? #642

@daegun
daegun commented Sep 9, 2016 edited

this problem is still present in rails 5. fortunately @jonleighton's fix still works.

@dracos dracos added a commit to dracos/rails that referenced this issue Sep 10, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.
66d03c5
@dracos
dracos commented Sep 10, 2016

PR to fix this, with test, at #26445

@pixeltrix I agree those possible structures are accurate, and are what I use (well, I never don't have a text/plain part :) ). If your inline image was totally superfluous, and could be hidden from the text/plain user (e.g. a footer image), then you could theoretically do mixed(alternative(plain, related(html, inline)), attachment), but better not do that currently.

@dracos dracos added a commit to dracos/rails that referenced this issue Sep 16, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.
fc067ca
@dracos dracos added a commit to dracos/rails that referenced this issue Sep 17, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.
1bc65d6
@dracos dracos added a commit to dracos/rails that referenced this issue Sep 17, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.
c9a606b
@dracos dracos added a commit to dracos/rails that referenced this issue Sep 17, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.
e82a2d3
@clemens clemens added a commit to clemens/rails that referenced this issue Sep 29, 2016
@dracos @clemens dracos + clemens Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.
df445a7
@dracos dracos added a commit to dracos/rails that referenced this issue Nov 23, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.

Thanks to clemens and eGust for reviewing.
2b1419a
@dracos dracos added a commit to dracos/rails that referenced this issue Nov 24, 2016
@dracos dracos Correctly wrap inline attachments.
This switches the behaviour from:

multipart/related
  multipart/alternative
    text/plain
    text/html
  attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

to:

multipart/mixed
  multipart/related
    multipart/alternative
      text/plain
      text/html
    attachment (disposition: inline, image1)
  attachment (disposition: attachment, file1)
  attachment (disposition: attachment, file2)

Fixes #2686.

Thanks to clemens and eGust for reviewing.
540faba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment