Add context to Splunk 'source'

[malvidin]

I recommend adding available email and/or file data for the Splunk 'source' for all processed files, if possible. Below is one possible option.

IMAP

• TA-dmarc/email@domain.org/domain.com!domain.org!123456.gz

Directory

• TA-dmarc/domain.com!domain.org!123456.gz

[jorritfolmer]

PR merged.

For setting the Splunk source field of imapped reports, I think we best combine imap2dir and dir2splunk into a new class "rfc822tosplunk".
This enables us to:

• set the appropriate source
• add relevant mail headers to a new rfc822 json object with stuff like rfc822.from, rfc822.subject, rfc822.dkim.domain, rfc822.dkim.domain.validation_result (issue Attacks on reporting URIs as mentioned in RFC 7489 #11)
• move from a batched imap -> dir flow, to a single flow by message
• re-use this new class to ingest maildir style dirs

[malvidin]

Does it matter which email RFC is mentioned (822, 2822, 5322)?

I do not have access to IMAP, so I cannot test any code against actual reports.

Should the dmarc code be maintained in a different repository, and merged into TA-dmarc periodically?

[jorritfolmer]

I committed some initial DKIM stuff in the rfc822tosplunk branch a while ago. It works with the current code and emits only log messages now.
While working on this I discoved that it is likely a bad idea to go for one giant class to handle everything. Too much duplication of code. So in input_module_dmarc_json_imap I added some commented out pseudocode just to see if multiple smaller classes would be a better idea. I think it is, but thats also where I left things becauce time.

Regarding IMAP testing: You can setup a local Dovecot imap server to which you copy some dmarc mails.