Add an email saver

[mavam]

This PR adds an email saver.

Tasks

Edit tasklist title

Beta Give feedback Tasklist Tasks, more options

• Rename
• Copy markdown
• Delete tasklist

Delete tasklist

Delete tasklist block?

Are you sure? All relationships in this tasklist will be removed.

Cancel Delete

1. Implement SMTP and SMTPS

   Implement SMTP and SMTPS
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
2. Write documentation

   Write documentation
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
3. Add changelog entry

   Add changelog entry
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove

Add item to Tasks

Review instructions

I tested this with this pipeline:

version
| repeat 3
| write json
| save email -f mavam@localhost -v user@example.org

For the first version of the email saver, I deliberately ignored MIME parts. Libcurl supports this nicely, but I'd rather get some experience with this before going more complex. Turns out that this was easier than expected. It's now part of the PR.

An open question is how to batch events. Today, we only have the batch operator. In TQLv2, we'll be able to window at the event level, which is the missing piece here. I think we can ignore this problem until then.

[mavam]

This works now for me. Here's how I tested this on macOS:

Install MailHog

brew install mailhog
brew services start mailhog
open http://127.0.0.1:8025

Setup Postfix

Add this to /etc/postfix/main.cf for MailHog:

myhostname = localhost
relayhost = [localhost]:1025

Then run:

sudo postfix stop
sudo postfix start

[mavam]

So I think this works well, except that my local test setup with MailHog struggles with multiple emails in the same SMTP session. Libcurl does the right thing though:

*   Trying [::1]:25...
* Connected to localhost (::1) port 25
< 220 localhost ESMTP Postfix
> EHLO ninja
< 250-localhost
< 250-PIPELINING
< 250-SIZE 10485760
< 250-VRFY
< 250-ETRN
< 250-ENHANCEDSTATUSCODES
< 250-8BITMIME
< 250-DSN
< 250 SMTPUTF8
> MAIL FROM:<mavam@localhost> SIZE=196
< 250 2.1.0 Ok
> RCPT TO:<user@example.org>
< 250 2.1.5 Ok
> DATA
< 354 End data with <CR><LF>.<CR><LF>
* We are completely uploaded and fine
< 250 2.0.0 Ok: queued as 0C92B7CC3E90
* Connection #0 to host localhost left intact
* Found bundle for host: 0x600000ffb900 [serially]
* Re-using existing connection with host localhost
> MAIL FROM:<mavam@localhost> SIZE=196                     <-------- this is legit
< 502 5.5.2 Error: command not recognized
* MAIL failed: 502
> QUIT
< 250 2.1.0 Ok
* Closing connection

I marked the part of the SMTP protocol that's legit. My hunch is that MailHog is simply not meant for this type of email testing.

[mavam]

My hunch is that MailHog is simply not meant for this type of email testing.

I was wrong. The actual reason was that I initially didn't terminate the email with <CR><LF>.<CR><LF>, but just .<CR><LF>. I found out about this only because I implemented the MIME part wrapping as well, and there MailHog accepted multiple emails per connection. So I knew it there was a bug in the other code path.

Anyway, it's now ready to go.

[tobim]

I tested this in an environment with a production postfix setup and it worked as expected with:

./tenzir 'version
  | put version
  | python "self.version = f\"Hello from Tenzir {self.version}, this email was sent via the tenzir email operator!\""
  | write lines
  | save email --subject "Tenzir Test" red@act.ed'