Single-part digest?

[squeakbat]

Not all e-mail clients (ahem, mail user agents) display the multipart digest format in a usable way, for example k-9 on android and gmail just show the parts as downloadable attachments.

It would be nice to have an optional digest format as a single (html or plaintext) body. It can basically look like the concatenation of the digest parts with some separator. Alternatively (the cheap and nasty version), we can just make the message multipart/mixed wrapped around text/html, instead of multipart/digest around message/rfc822 around text/html.

What do y'all think?

Since I want this feature, I'll be happy to write the code, obvs.

[devilgate]

I completely agree, and this is either a duplicate of #173, or at least closely related. I've started looking at that one a couple of times, but never quite got up the energy to work on it properly, so if you do make this change, I'd like to know.

[squeakbat]

Sorry about the delay in answering. (It came in when I had appendicitis (no kidding), then it got buried.)

It's a messy issue. I suppose we can add an option like "digest-one-part" and keep the old format as default. But given how few mail user agents can display multipart/digest message/rfc822, I'm really more inclined to just switch to a single-part digest format as the only option. It'll break compatibility with digest-post-process though.

The code change to support both formats is a little complicated, but doable (obviously). I think mostly feed._end_digest() will be more complicated.

I would like some feedback on this from maintainers before going forward. Thanks!

[kriswilk]

IMHO, this issue needs urgent action. rss2email's current digest format is objectively useless in gmail (and others by the sound of it). I appreciate wanting to be standards-compliant. But it's more important to be functionally useful. If that means reverting to a single-part digest, then so be it.

[xrat]

I am afraid it won't help anyone here but just in case: I wrote an Awk script that flattens r2e's multipart/digest: https://gist.github.com/xrat/d8b92e32e22af0cf8e05c1e7538cd2f5

[squeakbat]

OK ok. I shouldn't keep putting off something that should only take a couple hours to do.

[auouymous]

Would it be possible to translate that awk code to python in a post-processing script?

[xrat]

@auouymous My Python skills are sadly not yet sufficient.

[squeakbat]

This is my version. There's a bonus postproc function for adding feed name to the subject (for nondigest messages).

But this is only a stopgap. We should do it right in the first place.

import email.mime.multipart as mp
import email.header as h

# prepend feed name to subject
def addfeedname(feed, parsed, entry, guid, message):
   setsubject(message, parsed['feed']['title'] + ' ' + str(message['Subject']))
   return message

# turn multipart/digest into multipart/mixed
def merge(feed, parsed, seen, message):
   setsubject(message, parsed['feed']['title'] + ' digest')

   assert message.get_content_type() == "multipart/digest"
   parts = []
   for part in message.get_payload():
      assert part.get_content_type() == "message/rfc822"
      # XXX feed._end_digest() wants the same level of nesting as before,
      # so wrap the message in a singleton multipart/mixed
      #parts += part.get_payload()
      parts.append(mp.MIMEMultipart(_subparts = part.get_payload()))

   new = mp.MIMEMultipart(_subparts = parts)
   new['To'] = message['To']
   new['Subject'] = message['Subject']

   return new

def setsubject(message, s):
   del message['Subject']
   message['Subject'] = h.Header(s)

[squeakbat]

See pull request #221.

[squeakbat]

This is the cheap and nasty version alluded to in my original comment. Also, there's no option to get the original format.

It is, however, simple (and in fact shorter than the code it replaces).

[squeakbat]

Just to be clear, this is what the output looks like. It's a little weird that the parts have full message headers that are ignored.

As @kriswilk says above, the original format is more proper, but on the other hand it's also basically useless.

Content-Type: multipart/mixed; boundary="===============2677773025527576336=="
MIME-Version: 1.0
To: ...
Message-ID: <67348135-32ee-4e0c-976a-c73b5f07e8af@snake>
User-Agent: rss2email/3.13 (https://github.com/rss2email/rss2email)
List-ID: <apod.localhost>
List-Post: NO (posting not allowed on this list)
X-RSS-Feed: https://apod.nasa.gov/apod.rss
Subject: APOD digest
From: APOD <r2e@ewxyzzy.net>
Date: Thu, 23 Jun 2022 05:00:01 -0000

--===============2677773025527576336==
MIME-Version: 1.0
Content-Type: text/html; charset="us-ascii"
From: APOD <r2e>
To: ...
Content-Transfer-Encoding: 7bit
Date: Thu, 23 Jun 2022 05:00:01 -0000
Message-ID: <711c86d1-fd8e-4a70-84fd-c88ffdc0dac8@snake>
User-Agent: rss2email/3.13 (https://github.com/rss2email/rss2email)
List-ID: <apod.localhost>
List-Post: NO (posting not allowed on this list)
X-RSS-Feed: https://apod.nasa.gov/apod.rss
X-RSS-ID: 340337b212494788f2e7895d1cf5f88b91d155ca
X-RSS-URL: https://apod.nasa.gov/apod/ap220622.html
Subject: APOD Supernova Remnant: The Veil Nebula

...

--===============2677773025527576336==
MIME-Version: 1.0
Content-Type: text/html; charset="us-ascii"
From: APOD <r2e>
To: ...
Content-Transfer-Encoding: 7bit
Date: Thu, 23 Jun 2022 05:00:01 -0000
Message-ID: <c229630e-9f2a-4b80-9502-359311295ab8@snake>
User-Agent: rss2email/3.13 (https://github.com/rss2email/rss2email)
List-ID: <apod.localhost>
List-Post: NO (posting not allowed on this list)
X-RSS-Feed: https://apod.nasa.gov/apod.rss
X-RSS-ID: 2c120cbd5dc84c3b1c60071ccce6af01f0b7f5c5
X-RSS-URL: https://apod.nasa.gov/apod/astropix.html
Subject: APOD Spiral Galaxy NGC 6744

...

--===============2677773025527576336==--

[squeakbat]

Pull request merged. Happy ending for all?