Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Output encoding error in POSIX locale. #296

jplona opened this Issue May 7, 2012 · 3 comments


None yet
4 participants

jplona commented May 7, 2012

The full error message is

Server error: export.tex: commitBuffer: invalid argument (invalid character)

I'm using commit a9e422b6ea05297dd931c606edb313be2392dcac, built with cabal install --prefix=/usr/local, on an up-to-date Arch Linux installation.

The point of the error is line 245 of Network/Gitit/Export.hs, which reads writeFile tempfile latex.

A little googling led me to believe this is an output encoding issue, so I changed my system locale to en_US.UTF-8 which rectified the problem. Nevertheless, since Pandoc assumes that all input and output is UTF-8, files written for consumption by Pandoc should always be encoded in UTF-8 regardless of the system locale.

The instigator of the problem was the euro character (Hex: e282ac) included in the default LaTeX template, located at /usr/share/pandoc1.9.2/templates/default.latex on my machine.

In the non-working locale, the value of System.IO.localeEncoding is ANSI_X3.4-1968. The output from locale reads POSIX for all non-empty values. In the working locale, POSIX is replaced by en_US.UTF-8.

How about adding

import Data.ByteString.Lazy.UTF8 (toString, fromString)

-- | Write string to file as UTF-8.  Encode filename as UTF-8.
writeFileUTF8 :: FilePath -> String -> IO ()
#if MIN_VERSION_base(4,5,0)
writeFileUTF8 f s = B.writeFile f $ fromString s
writeFileUTF8 f s = B.writeFile $ encodeString f $ fromString s

to Utils.hs?

By the way, why putting an euro sign in pandoc template default.latex ?


jgm commented Apr 21, 2013

+++ linquize [Apr 21 13 03:02 ]:

By the way, why putting an euro sign in pandoc template default.latex ?

The euro is a common character, and ought to work by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment