Encoding UTF-8 #42

lightmagic1 · 2019-04-19T20:28:18Z

mailmerge_template.txt


TO: {{email}}
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
Content-Type: text/html; charset="utf-8"

<!doctype html>
<html lang="pt-br">

<head>
    <title> Page title </title>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <style type="text/css">
        * {
            margin: 0;
            border: 0;
            padding: 0;
            color: red;

            font-family: Arial, Helvetica, sans-serif;
        }
    </style>
</head>

<body>

<h1>ççç ÇÇÇ áéíóú !.</h1>
</body>

</html>

Terminal:

>mailmerge
>>> message 0
TO: light.magic1@gmail.com
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
Content-Type: text/html; charset="utf-8"

<!doctype html>
<html lang="pt-br">

<head>
    <title> Page title </title>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <style type="text/css">
        * {
            margin: 0;
            border: 0;
            padding: 0;
            color: red;

            font-family: Arial, Helvetica, sans-serif;
        }
    </style>
</head>

<body>

<h1>Ã§Ã§Ã§ Ã‡Ã‡Ã‡ Ã¡Ã©ÃÃ³Ãº !.</h1>
</body>

</html>
>>> encoding utf-8
>>> sent message 0 DRY RUN
>>> No attachments were sent with the emails.
>>> Limit was 1 messages.  To remove the limit, use the --no-limit option.
>>> This was a dry run.  To send messages, use the --no-dry-run option.

Recieved mail:

Am i doing anything wrong? What should i do to those characters appears?

The text was updated successfully, but these errors were encountered:

awdeorio · 2019-04-20T11:03:57Z

Here's a couple things to try.

Check your Python version. I expect Python 3.

$ python --version

Check the encoding of the template. I expect UTF-8.

$ file mailmerge_template.txt

lightmagic1 · 2019-04-21T04:29:52Z

Here's a couple things to try.

Check your Python version. I expect Python 3.
$ python --version
Check the encoding of the template. I expect UTF-8.
$ file mailmerge_template.txt

λ python --version
Python 3.7.3

λ file mailmerge_template.txt
mailmerge_template.txt: HTML document, UTF-8 Unicode text, with CRLF line terminators

anything else that i should try?

:(

awdeorio · 2019-04-21T11:24:08Z

Digging a little deeper, it looks like this is not related to actually sending the message. On my machine, I get the expected output at the terminal.

$ python3 --version
Python 3.7.2
$ file mailmerge_*
mailmerge_database.csv: ASCII text
mailmerge_server.conf:  ASCII text
mailmerge_template.txt: HTML document text, UTF-8 Unicode text
$ mailmerge 
>>> message 0
TO: myself@mydomain.com
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
Content-Type: text/html; charset="utf-8"

<!doctype html>
<html lang="pt-br">

<head>
    <title> Page title </title>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <style type="text/css">
        * {
            margin: 0;
            border: 0;
            padding: 0;
            color: red;

            font-family: Arial, Helvetica, sans-serif;
        }
    </style>
</head>

<body>

<h1>ççç ÇÇÇ áéíóú !.</h1>
</body>

</html>

>>> encoding utf-8
>>> sent message 0 DRY RUN
>>> Limit was 1 messages.  To remove the limit, use the --no-limit option.
>>> This was a dry run.  To send messages, use the --no-dry-run option.

Next thing to check is the file encoding of the database.

lightmagic1 · 2019-04-22T19:27:37Z

mailmerge_database.csv: ASCII text, with CRLF line terminators
mailmerge_server.conf: ASCII text, with CRLF line terminators
mailmerge_template.txt: HTML document, UTF-8 Unicode text, with CRLF line terminators

awdeorio · 2019-04-23T11:25:55Z

That looks OK to me. Next thing to try: use the smallest email you can to try and reproduce the bug. Something like this:

mailmerge_template.txt

TO: {{email}}
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
ççç ÇÇÇ áéíóú

lightmagic1 · 2019-04-26T06:19:04Z

TO: {{email}}
SUBJECT: Testing mailmerge
FROM: My Self myself@mydomain.com
ççç ÇÇÇ áéíóú

mailmerge_template.txt

TO: {{email}}
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
ççç ÇÇÇ áéíóú

Output

λ mailmerge --no-dry-run
>>> message 0
TO: light.magic1@gmail.com
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
Ã§Ã§Ã§ Ã‡Ã‡Ã‡ Ã¡Ã©ÃÃ³Ãº
>>> encoding utf-8
>>> Read SMTP server configuration from mailmerge_server.conf
>>>   host = smtp.gmail.com
>>>   port = 465
>>>   username = light.magic1@gmail.com
>>>   security = SSL/TLS
>>> password for light.magic1@gmail.com on smtp.gmail.com:
>>> sent message 0
>>> No attachments were sent with the emails.
>>> Limit was 1 messages.  To remove the limit, use the --no-limit option.

Message in my inbox:

awdeorio · 2019-04-26T10:39:16Z

Let's try it without mailmerge, just with Python.

$ cp mailmerge_template.txt test.py

Edit test.py to look like this:

print("ççç ÇÇÇ áéíóú")

Now run it.

$ file test.py
test.py: UTF-8 Unicode text
$ python3 test.py 
ççç ÇÇÇ áéíóú

lightmagic1 · 2019-04-26T18:55:46Z

print("ççç ÇÇÇ áéíóú")

λ cp mailmerge_template.txt test.py

λ file test.py
test.py: UTF-8 Unicode text, with no line terminators

λ python3 test.py
ççç ÇÇÇ áéíóú

awdeorio · 2019-04-26T21:10:57Z

Well, looks like that's not it. Is mailmerge using Python2 or Python3?

$ which mailmerge 
/usr/local/bin/mailmerge
$ head /usr/local/bin/mailmerge
#!/usr/local/opt/python/bin/python3.7
# -*- coding: utf-8 -*-
import re
import sys

from mailmerge.__main__ import cli

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(cli())

lightmagic1 · 2019-04-26T22:10:55Z

Well, looks like that's not it. Is mailmerge using Python2 or Python3?

$ which mailmerge 
/usr/local/bin/mailmerge
$ head /usr/local/bin/mailmerge
#!/usr/local/opt/python/bin/python3.7
# -*- coding: utf-8 -*-
import re
import sys

from mailmerge.__main__ import cli

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(cli())

Anaconda CondaEnv Python 3.7.3

λ pip list
Package       Version
------------- --------
backports.csv 1.0.7
certifi       2019.3.9
chardet       3.0.4
Click         7.0
configparser  3.7.4
future        0.17.1
Jinja2        2.10.1
mailmerge     1.8
MarkupSafe    1.1.1
pip           19.0.3
setuptools    41.0.0
wheel         0.33.1
wincertstore  0.2

λ which mailmerge
/c/Anaconda3/envs/qrcode/Scripts/mailmerge

λ head /c/Anaconda3/envs/qrcode/Scripts/mailmerge
Python 3.7.3 (default, Mar 27 2019, 17:13:21) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32

>>> #!/usr/local/opt/python/bin/python3.7
>>> # -*- coding: utf-8 -*-
>>> import re
>>> import sys
>>> from mailmerge.__main__ import cli
ççç ÇÇÇ áéíóú
>>> if __name__ == '__main__':
...      sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
...      sys.exit(cli())
...
>>> message 0
TO: light.magic1@gmail.com
SUBJECT: Testing mailmerge
FROM: My Self <myself@mydomain.com>
Ã§Ã§Ã§ Ã‡Ã‡Ã‡ Ã¡Ã©ÃÃ³Ãº
>>> encoding utf-8
>>> sent message 0 DRY RUN
>>> No attachments were sent with the emails.
>>> Limit was 1 messages.  To remove the limit, use the --no-limit option.
>>> This was a dry run.  To send messages, use the --no-dry-run option.

That's weird o_o

awdeorio · 2019-04-30T11:48:21Z

Mailmerge works with either Python2 or Python3. Next thing to try: let's see if Anaconda is causing a problem. We'll install mailmerge from source, using the absolute path to a non-Anaconda python executable. If you're on Windows, use WSL or Cygwin. If you're on OSX, use homebrew to install Python.

Make sure that your Python executable is not Anaconda. Make sure that PYTHONPATH is not set.

$ which python3
/usr/local/bin/python3
$ echo $PYTHONPATH
# <blank>

Get the source code

$ git clone https://github.com/awdeorio/mailmerge.git
Cloning into 'mailmerge'...

Install using the absolute path to Python 3. Install virtual environment with a local copy of pip.

$ /usr/local/bin/python3 -m venv venv
$ source venv/bin/activate
$ which pip
/Users/awdeorio/Downloads/mailmerge/venv/bin/pip
$ pip install -e .

Verify that you have a local copy of mailmerge installed.

$ which mailmerge
/Users/awdeorio/Downloads/mailmerge/venv/bin/mailmerge
$ mailmerge --version
mailmerge, version 1.9

Now, try your test again.

awdeorio · 2019-06-12T18:01:46Z

Looks like this issue has gone stale. Please reopen if you have more information!

awdeorio closed this as completed Jun 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Encoding UTF-8 #42

Encoding UTF-8 #42

lightmagic1 commented Apr 19, 2019

awdeorio commented Apr 20, 2019

lightmagic1 commented Apr 21, 2019

awdeorio commented Apr 21, 2019

lightmagic1 commented Apr 22, 2019

awdeorio commented Apr 23, 2019

lightmagic1 commented Apr 26, 2019

awdeorio commented Apr 26, 2019

lightmagic1 commented Apr 26, 2019

awdeorio commented Apr 26, 2019

lightmagic1 commented Apr 26, 2019

awdeorio commented Apr 30, 2019 •

edited

Loading

awdeorio commented Jun 12, 2019

Encoding UTF-8 #42

Encoding UTF-8 #42

Comments

lightmagic1 commented Apr 19, 2019

awdeorio commented Apr 20, 2019

lightmagic1 commented Apr 21, 2019

awdeorio commented Apr 21, 2019

lightmagic1 commented Apr 22, 2019

awdeorio commented Apr 23, 2019

lightmagic1 commented Apr 26, 2019

awdeorio commented Apr 26, 2019

lightmagic1 commented Apr 26, 2019

awdeorio commented Apr 26, 2019

lightmagic1 commented Apr 26, 2019

awdeorio commented Apr 30, 2019 • edited Loading

awdeorio commented Jun 12, 2019

awdeorio commented Apr 30, 2019 •

edited

Loading