special unicode chars causes UnicodeDecodeError in offline mode

[sqlalchemy-bot]

Migrated issue, originally created by bbinet (@bbinet)

Traceback (most recent call last):
  File "/home/bbinet/dev/SiaCG62/buildout/bin/alembic", line 20, in <module>
    alembic.config.main()
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/config.py", line 255, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/config.py", line 250, in main
    self.run_cmd(cfg, options)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/config.py", line 241, in run_cmd
    **dict((k, getattr(options, k)) for k in kwarg)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/command.py", line 148, in downgrade
    script.run_env()
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/script.py", line 191, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/util.py", line 185, in load_python_file
    module = imp.load_source(module_id, path, open(path, 'rb'))
  File "/home/bbinet/dev/SiaCG62/siacg62/alembic/env.py", line 73, in <module>
    run_migrations_offline()
  File "/home/bbinet/dev/SiaCG62/siacg62/alembic/env.py", line 41, in run_migrations_offline
    context.run_migrations()
  File "<string>", line 7, in run_migrations
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/environment.py", line 494, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/migration.py", line 211, in run_migrations
    change(**kw)
  File "/home/bbinet/dev/SiaCG62/siacg62/alembic/versions/201211191711_17a6073d49ac_listeplanue_values_s.py", line 55, in downgrade
    ll.c.id_liste_child==_listeid_select('ListePlanUE', valeur))))
  File "<string>", line 7, in execute
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/operations.py", line 813, in execute
    execution_options=execution_options)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/ddl/impl.py", line 78, in execute
    self._exec(sql, execution_options)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/ddl/impl.py", line 70, in _exec
    ).replace("\t", "    ").strip() + self.command_terminator)
  File "/home/bbinet/dev/SiaCG62/buildout/eggs/alembic-0.4.0-py2.6.egg/alembic/ddl/impl.py", line 53, in static_output
    self.output_buffer.write(text + "\n\n")
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 330: ordinal not in range(128)

By default, in offline mode, sql will be written to sys.stdout, so we hit the following problem in python 2.6:
http://stackoverflow.com/questions/8016236/python-unicode-handling-differences-between-print-and-sys-stdout-write

[sqlalchemy-bot]

bbinet (@bbinet) wrote:

Note that you can easily work around this issue by configuring the output_buffer in your context. For example you can update the run_migrations_offline function in your env.py file so that it looks like:

import sys, codecs

def run_migrations_offline():
    url = config.get_main_option("sqlalchemy.url")
    wrapped_stdout = codecs.getwriter('UTF-8')(sys.stdout)
    context.configure(url=url, output_buffer=wrapped_stdout)
    with context.begin_transaction():
        context.run_migrations()

[sqlalchemy-bot]

Changes by bbinet (@bbinet):

• edited description

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

I've added output_encoding as a configure() argument in 0b316ec.

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed

[sqlalchemy-bot]

bbinet (@bbinet) wrote:

Thanks.