Add support for composing 'multipart/related' emails #3197

dcpurton · 2022-01-15T14:57:56Z

What does this PR do?

Adds support for composing multipart/related emails.

Example

run ./neomutt -n -F multipart-related.rc

Needs:

`multipart-related.rc`

# vim: syn=neomuttrc

# filename: multipart-related.rc

# run from build directory:
#   ./neomutt -n -F multipart-related.rc

# key in <F1> through <F6> to construct the email

unset record
unset signature

set attach_format = "%u%D%I %t%4n %T%.40d%> [%.15m/%.15M, %.6e%?C?, %C?, %s]"
set editor = "echo '![Neomutt Logo](cid:neomutt-256.png)' >> %s"
set from = "john.doe@example.com"
set help = no
set postpone = no
set postponed = "postponed.mbox"
set realname = "John Doe"
set recall=no
set spoolfile = "inbox.mbox"

macro index   <F1> "<mail>John Doe <john.doe@example.com><enter>test<enter>"
macro compose <F2> "<attach-file>contrib/logo/neomutt-256.png<enter>"
macro compose <F3> "<rename-attachment><enter><toggle-disposition>"
macro compose <F4> "<attach-file>html-part.html<enter>"
macro compose <F5> "<toggle-disposition><tag-entry>1<enter><tag-entry><group-alternatives>"
macro compose <F6> "<tag-entry>4<enter><tag-entry><group-related>"

`html-part.html`

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="generator" content="pandoc">
  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
  <title> </title>
</head>
<body>
<div style="max-width: 15cm;">
<p style="margin: 1ex 0; padding: 0;"><img src="cid:neomutt-256.png" alt="Neomutt Logo"></p>
</div>
</body>
</html>

Final email structure

  I     1 <no description>                                                   [multipart/related, 7bit, 0K]
  I     2 ├─><no description>                                            [multipart/alternative, 7bit, 0K]
- I     3 │ ├─>/tmp/neomutt-hostname-XXXX-XXXXXX-XXXXXXXXX              [text/plain, 7bit, us-ascii, 0.1K] 
  I     4 │ └─>html-part.html                                          [text/html, 8bit, iso-8859-1, 0.4K] 
  I     5 └─>neomutt-256.png                                                      [image/png, base64, 19K]

Behaviour at receiving end

MUA	PGP: Clear	PGP: Signed	PGP: Signed and Encrypted
Gmail (Android)	✔️	✔️	n/a
Gmail (Android, Exchange)	✔️¹	❌²	n/a
Gmail (iOS)	✔️	✔️	n/a
Gmail (Web)	✔️	✔️	n/a
Mail App (Windows)	✔️	✔️	n/a
Outlook (Android)	✔️	✔️	n/a
Outlook (Web)	✔️	✔️	n/a
Outlook (iOS)	✔️	✔️	n/a
Outlook (Web)	✔️	✔️	n/a
Outlook (Windows)	✔️	✔️	n/a
Proton (Android)	✔️	❌³	❌³
Proton (iOS)	✔️	✔️⁴	✔️⁴
Proton (Web)	✔️	✔️	✔️
Thunderbird (Linux)	✔️	✔️	✔️

¹ Also displays embedded image as attachment (but emails from other clients display like this too).
² Embedded image is not displayed and isn't available in attachment list. (Also fails on messages from Thunderbird.)
³ Embedded image is not displayed, but is available in attachment list. (Also fails on messages from Thunderbird.)
⁴ Thinks there is remote content for some reason.

Notes on constructing emails

Attachments in the multipart/related group should probably have inline disposition. (Microsoft Outlook shows also shows them as a separate attachment if they have disposition attachment, but Gmail doesn't.) So ensure embedded images have the header Content-Disposition: inline; filename="image.png". The attachment needs to be given a d_filename (with <rename-attachment>) for the filename field to be included.
Actual attachments should be placed outside of the multipart/related group.

flatcap · 2022-01-15T16:06:37Z

It works!

dcpurton · 2022-01-16T09:06:17Z

Using a Neomutt macro to pipe the primary attachment (in Markdown format) to a sufficiently complex script does allow you to construct and send a multipart/alternative email wrapped in a multipart/related group if required.

Consider initial email structure like this:

- I     1 /tmp/neomutt-hostname-XXXX-XXXXXX-XXXXXXXXXX                  [text/plain, 7bit, us-ascii, 0.1K]
  A     2 image.png                                                               [image/png, base64, 19K]
  A     3 attachment.pdf                                                   [application/pdf, quoted, 7.1K]

image.png can be referenced in the main file as:

![Alt Text](cid:image.png)

We can send the message with a single macro:

macro compose y "<first-entry><enter-command>set wait_key=no<enter><pipe-entry>msg2html.sh<enter>
  <attach-file>/tmp/neomutt-alternative.html<enter><toggle-unlink><toggle-disposition><tag-entry>
  <first-entry><tag-entry><group-alternatives><enter-command>set wait_key=yes<enter>
  <enter-command>source /tmp/neomutt.rc<enter><send-message>"

msg2html.sh produces /tmp/neomutt-alternative.html using pandoc and parses the Markdown looking for (cid:filename) and creates /tmp/neomutt.rc with the commands to tag the appropriate files and create the related/multipart group. If no (cid:filename) is found, then leave /tmp/neomutt.rc empty.

In the above case /tmp/neomutt.rc would contain:

push "<group-related>"
push "<search>image.png<enter><rename-attachment>image.png<enter><toggle-disposition><tag-entry>"
push "<tag-entry>"

The final email structure before sending will be:

  I     1 <no description>                                                   [multipart/related, 7bit, 0K]
  I     2 ├─><no description>                                            [multipart/alternative, 7bit, 0K]
- I     3 │ ├─>/tmp/neomutt-hostname-XXXX-XXXXXX-XXXXXXXXXX             [text/plain, 7bit, us-ascii, 0.1K]
- I     4 │ └─>/tmp/neomutt-alternative.html                                [text/html, 8bit, utf-8, 0.6K]
  I     5 └─>image.png                                                            [image/png, base64, 19K]
  A     6 attachment.pdf                                                   [application/pdf, quoted, 7.1K]

At send time Neomutt then wraps all this in a multipart/mixed group and signs/encrypts it if required.

And provided you haven't forgot to attach anything, misspelled anything, or omitted the cid: … it will all work fine at the receiving end.

flatcap

Rebased
Clang-formatted
A couple of minor tweaks to the docs

Looking good

send/header.c

compose/functions.c

email/parse.c

problems fixed

flatcap · 2022-01-26T03:18:35Z

Rebased over a bunch of huge Compose-function renames.

- Reads from and writes into the Body parameter list.

flatcap

Oh! It was going so well... and you blew it :-)
It doesn't work; the Content-ID that gets sent is garbage.

However, I've pushed some fixes.

In mutt_write_mime_header(), you id = cont->attribute; but you actually wanted cont->value

Just below that, pl_conts gets freed: mutt_param_free(&pl_conts); which means that id is no longer valid. I've added a strdup() and free() to work around this.

I've added a mutt_buffer_dealloc() to check_cid() to cover the possibility of an error.

Finally, in the docs it says: "an HTML part ... needs to contain ... <img src="cid:content-id">"
I've removed the reference to alt="Alt Text" as that's not required.

dcpurton · 2022-01-27T22:23:10Z

Oh! It was going so well... and you blew it :-) It doesn't work; the Content-ID that gets sent is garbage.

Oops! It definitely worked at one stage.

However, I've pushed some fixes.

Great.

Finally, in the docs it says: "an HTML part ... needs to contain ... <img src="cid:content-id">" I've removed the reference to alt="Alt Text" as that's not required.

Fine.

flatcap added type:discuss Your views/opinions are requested type:enhancement Feature Request labels Jan 15, 2022

dcpurton changed the title ~~Add support for 'multipart/related' emails~~ Add support for composing 'multipart/related' emails Jan 17, 2022

dcpurton force-pushed the devel/multipart-related branch 2 times, most recently from adffaee to 0a0f353 Compare January 18, 2022 06:58

dcpurton marked this pull request as ready for review January 19, 2022 12:36

dcpurton requested a review from flatcap January 19, 2022 12:40

flatcap force-pushed the devel/multipart-related branch from ea1dc6c to f6e0c1a Compare January 19, 2022 14:12

flatcap previously requested changes Jan 19, 2022

View reviewed changes

send/header.c Show resolved Hide resolved

compose/functions.c Outdated Show resolved Hide resolved

compose/functions.c Outdated Show resolved Hide resolved

email/parse.c Show resolved Hide resolved

email/parse.c Outdated Show resolved Hide resolved

dcpurton force-pushed the devel/multipart-related branch from f6e0c1a to 1ee148b Compare January 19, 2022 22:52

flatcap force-pushed the devel/multipart-related branch from 1ee148b to c330bad Compare January 26, 2022 03:15

dcpurton force-pushed the devel/multipart-related branch 3 times, most recently from 14fa3a2 to a2c6cec Compare January 27, 2022 13:24

dcpurton added 7 commits January 27, 2022 18:15

email: support MIME Content-ID header

7ef0602

- Reads from and writes into the Body parameter list.

compose: add gen_cid() function

0730dbe

compose: add check_cid() function to test for a valid Content-ID

80291c1

compose: support editing Content-ID of an attachment

16669ba

Allow d_filename with Content-Disposition: inline

419801a

compose: support multipart/related groups

4ab07eb

docs: document composing 'multipart/related' emails

5dd9fe2

flatcap force-pushed the devel/multipart-related branch from a2c6cec to 5dd9fe2 Compare January 27, 2022 19:41

flatcap approved these changes Jan 27, 2022

View reviewed changes

flatcap merged commit 77b59af into master Jan 27, 2022

flatcap deleted the devel/multipart-related branch January 31, 2022 12:23

flatcap assigned dcpurton Aug 19, 2023

gahr mentioned this pull request Jun 13, 2024

RFC2231-style MIME part header continuations erroneously used for long Content-ID fields #4327

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for composing 'multipart/related' emails #3197

Add support for composing 'multipart/related' emails #3197

dcpurton commented Jan 15, 2022 •

edited

Loading

flatcap commented Jan 15, 2022

dcpurton commented Jan 16, 2022 •

edited

Loading

flatcap left a comment

flatcap commented Jan 26, 2022

flatcap left a comment

dcpurton commented Jan 27, 2022

Add support for composing 'multipart/related' emails #3197

Add support for composing 'multipart/related' emails #3197

Conversation

dcpurton commented Jan 15, 2022 • edited Loading

What does this PR do?

Example

Needs:

multipart-related.rc

html-part.html

Final email structure

Behaviour at receiving end

Notes on constructing emails

flatcap commented Jan 15, 2022

dcpurton commented Jan 16, 2022 • edited Loading

flatcap left a comment

Choose a reason for hiding this comment

flatcap commented Jan 26, 2022

flatcap left a comment

Choose a reason for hiding this comment

dcpurton commented Jan 27, 2022

dcpurton commented Jan 15, 2022 •

edited

Loading

`multipart-related.rc`

`html-part.html`

dcpurton commented Jan 16, 2022 •

edited

Loading