List names Encoding of German Umlauts ä ö ü Ä Ö Ü and possibly other characters #472

Open
frederickjh opened this Issue Feb 16, 2015 · 10 comments

Projects

None yet

2 participants

@frederickjh

Hi Justin!

I have see a number of issues related to encoding but I am not sure if my issue is related to them or what has been solved in the mean time. Here are the details of my current Dadamail setup. Let me know if there is anything else that would be helpful to know.

I am using Dadamail 7.4.0 Stable 12/16/14.
Server Software: Apache
Operating System: linux
Perl Version: 5.008008
In Mail Sending » Advanced Options, the options are :

Default Character Set : UTF8 UTF8
Default Plain Text Message Encoding : quoted-printable
Default HTML Message Encoding : quoted-printable
Encode Message Headers is checked

We are using the bridge plugin to send the messages. At this point I am not seeing any problems with the email messages themselves. They seem to be passing through with the correct encoding.

What I am having a problem with is fields in profiles and the list names. I am filing this issue on the list names as it is the most important to solve. If I enter a list name containing the German umlauts like so:
screenshot from 2015-02-16 13 27 58-list_name_entered_with_umlauts

After saving I end up with:
screenshot from 2015-02-16 13 29 01-list_name_entered_with_umlauts_results_after_save

When I send an email to the list using the bridge plugin it gets it half right (header correct, body text wrong) and send me back an email that looks like this:

screenshot from 2015-02-16 13 34 39-list_name_with_umlauts-bridge_reply

The list email is sent out and looks like this:

screenshot from 2015-02-16 13 35 59-list_name_with_umlauts-message_to_list

The umlauts are presented/encoded wrong.

If I enter the Umlauts using Unicoding (http://www.periodni.com/unicode_utf-8_encoding.html) you can see in the image that the title and select list are correct, but the input field remains in the Unicoding:

screenshot from 2015-02-16 13 31 41-list_name_entered_with_umlauts_using_unicode_encoding

If a message is sent out from the list both the bridge and messages sent to the list send the Unicoding out as it is.

screenshot from 2015-02-16 13 57 17-list_name_with_umlauts-unicoding_in_mail_header

Now if I am in the List information screen after having entered a list name with umlauts and saved and do not make any changes to the list name and then click "Save Changes" and interesting thing happens that maybe points to part of the issue.

This:
screenshot from 2015-02-16 13 29 01-list_name_entered_with_umlauts_results_after_save

Becomes this:

screenshot from 2015-02-16 14 05 21-list_name_with_umlauts-after_2nd_save

Clicking once more on "Save Changes" (even though we have not made any) results in this:

screenshot from 2015-02-16 14 06 06-list_name_with_umlauts-after_3rd_save

Is it re-encoding the already wrongly encoded text?

At this point I would just be happy to see this come through correctly in the emails as I am the only one using the admin interface. If that ended up correct too that would be great.

Thanks in advance for any help you can give!

Frederick

@justingit
Owner

In Dada Mail, what's the backend that you're using?

@frederickjh

MySQL
screenshot from 2015-02-16 23 22 07-phpmyadmin_database_tables

@justingit
Owner

I'll have to look over things in more detail - I'm not seeing the same problems you are.

I noticed you made some slight changes to the admin template - did you set a charset in the HTML, that's not UTF-8? Is there a specific browser that you're using?

@frederickjh

Sorry it has taken so long to get back to you justingit. I must have missed you message. I basically changed the logo the encoding is still UTF-8.
screenshot from 2015-03-18 08 28 26

@justingit
Owner

I really appreciate all the feedback you're giving. Attached is what I see, if I try to change the list name to, "Umlauts ä ö ü Ä Ö Ü "

your_mailing_list_ list_information-umlauts_a_o_u_a_o_u-dada_mail-_2015-03-18_09 14 22 png

Which looks great! But still doesn't recreate the problem you're having. I am using a slightly newer version of Dada Mail than you - this is the beta for version 8. If you have time, would you be able to test this version, and see if the problem has disappeared?

http://dadamailproject.com/support/documentation-8_0_0-beta1/install_dada_mail.pod.html

@frederickjh

I have upgraded to 9.1.1 and thought I would have a look and see when the support for German umlauts is at in Dada Mail

In Config.pm $HTML_CHARSET //= 'UTF-8';
SENDING > ADVANCED OPTIONS
Default Character Set = UTF-8 UTF-8
Default Plain Text Message Encoding = 8bit
Default HTML Message Encoding = 8bit
Encode message headers is ticked

I tested sending via the online wysiwyg editor and the result was good to the list with both the subject and text being displayed properly.
screenshot from 2016-02-26 01 09 11

However the subject of the message to the list owner did not fare as well.

screenshot from 2016-02-26 01 10 46

But the attached message that was sent to the list a bit below in the same email displays both subject and text properly.

screenshot from 2016-02-26 01 18 39

@frederickjh

The next test was with the bridge plugin which we are using mostly to send emails.

The Message Received email does not properly display the subject of the email received in the body of the text.

screenshot from 2016-02-26 01 25 39

The message sent out to the list subject displays properly except that a utf-8 is being added between the list name and the subject. The body displays correctly.

screenshot from 2016-02-26 01 26 09

I hope that this information helps to find and fix this issue. I still need to test umlauts in a list name.

@frederickjh

I changed my Test list name to Testing umlauts Ä Ö Ü ä ö ü - for this test.

Email sent from the web interface the subject looked like this. The body text also displayed correctly.

screenshot from 2016-02-26 01 53 27

The email to the list owner stating that the mailing is finished has the same problems as before but the list name at the beginning displays properly but title of the email does not, as before.

screenshot from 2016-02-26 01 54 31

@frederickjh

Testing with the bridge module results are the same as with sending via the web interface. I did discover another place where the umlauts are not working in the Not allowed to post email body text where the list name is given. Interestingly the list name is correct in the From header.

screenshot from 2016-02-26 02 15 44

Well I am finished testing this for now!

Thanks again for all your hard work on Dada Mail!
Frederick

@frederickjh

Hi!

Just an update. I am still seeing the name of the encoding being placed between the list name and the subject, but it is not always. Sometimes it does and sometimes not. Two emails from the same person, one does it and the other not. Screenshot follows.
Frederick

screenshot from 2016-03-14 17 36 59

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment