Bounces: unknown charset: message

[priffo-karrott]

OS: Linux Mint.
SMTP: Mandrillapp.
POP: outlook.

I setted bounces pop with outlook email. So i have the next issue in logs: error scanning bounce mailbox: unknown charset: unknown charset: message: unhandled charset "iso-8859-1"

If i used gmail, pop. I have: error scanning bounce mailbox: EOF.

In the same cases, the bounce email arrives.

Can you help me?

[knadh]

hm, looks like an issue with the POP3 mailbox scanner and this particular character encoding. This will need to be investigated and debugged.

[an0nfunc]

I also got this problem. Fixed by importing github.com/emersion/go-message/charset in

listmonk/internal/bounce/mailbox/pop.go

Line 8 in d42c676

"github.com/emersion/go-message"

import (
	"encoding/json"
	"regexp"
	"time"

	"github.com/emersion/go-message"
	_ "github.com/emersion/go-message/charset"
	"github.com/knadh/go-pop3"
	"github.com/knadh/listmonk/models"
)

I tested this and the error is gone and bounces work as expected.

[knadh]

@jaredfolkins thanks! That indeed is the fix.

[redispade]

I just updated and I still have this issue. v2.1.0 docker

app_1 | 2022/02/12 22:51:17 bounce.go:139: error scanning bounce mailbox: unknown charset: unknown charset: charset "us_ascii": htmlindex: invalid encoding name

[an0nfunc]

us_ascii is a weird encoding, and it's not supported by golang.org/x/text/encoding/charmap. (Apparently that is how ASCII should be called now.) What server do you use?

[redispade]

I use Mailu as smtp/pop, but it happens only with one account. I set another one and it works(inbox got emptied) but it didn't detect any bounces.

[an0nfunc]

Probably coming from a mail witch itself comes from some non-UTF-8-friendly source. I checked upstream (github.com/emersion/go-message) and they seem to have setup a method for handling such quirks already. Maybe you should open an issue there, this is probably out of scope for listmonk to handle. But I would wait until @knadh has had a look.

EDIT: What is weird is that us_ascii is listed in IANA's alias table here, which golang.org/x/text/encoding/ianaindex (us-ascii) is based on (and which is used by go-message), so it should be handled, but isn't. Not sure yet how that happens.

[knadh]

Ah, that's a bummer ;( I'd tested this with multiple POP3 boxes including Gmail and Outlook. Any chance I get access to a POP box where this issue's happening? I can try and fix this quickly and release an RC so that you don't have to wait for the next versioned release.

[redispade]

Ah, that's a bummer ;( I'd tested this with multiple POP3 boxes including Gmail and Outlook. Any chance I get access to a POP box where this issue's happening? I can try and fix this quickly and release an RC so that you don't have to wait for the next versioned release.

May I contact you via email (the one in your profile)?

EDIT:
Today it actually worked. I found the mailbox empty. I don't know how, because I didn't delete any message.
I would suggest to add a feature(like a checkbox) to not delete all messages from the bounce mailbox, just the bounce messages.

[knadh]

May I contact you via email (the one in your profile)?

Yep.

I would suggest to add a feature(like a checkbox) to not delete all messages from the bounce mailbox, just the bounce messages.

That's not feasible unfortunately. POP doesn't have a state management system like IMAP to remember which mails have been read and which haven't been. If emails are not deleted, they will have to be scanned over and over. That's why bounce mailboxes should be dedicated and should not receive non-bounce emails.

[Gahrt]

hey,

i have the same Error: error scanning bounce mailbox: EOF on the Version: v2.1.0 on a MS Exchange Mailbox which I use only for bounces

[ldidry]

@knadh I can give you access to a mailbox with a very similar problem:

error scanning bounce mailbox: unknown charset: unknown charset: charset "iso859-1": htmlindex: invalid encoding name

It should be iso8859-1.

NB: I had the problem in 2.1.0, still have it with 2.2.0.

[knadh]

@ldidry could you please e-mail me a set of test credentials for the inbox? kailash [at] nadh.in

[ldidry]

Credentials sent 🙂

[ldidry]

Hello @knadh, did you had some time to test the credentials?

[knadh]

Haven't had the time ;( Swamped.