You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've attached a patch to convert the source code to unicode via the source code's magic encoding comment (like Myghty does -- in fact I pretty much ripped all of dairiki's code for this). In the rare case that the magic encoding comment doesn't describe the non-ascii characters correctly, the 'replace' error handler is used to add garbage in its place
But before we commit it, I think I would like to see an addition to the Template class.
Pylons currently does this in its exception handler to render the exception:
Mike I may have asked you this before, but is there a reason why mako Templates don't have an 'encoding_errors' parameter to go along with 'output_encoding'?
If it did, we could have html_error_template set encoding_errors='htmlentitiyreplace'. Then anyone needing an html_error_template wouldn't have to be concerned with unicode at all -- they could simply render() (like Pylons is doing now) and always get back a sys.getdefaultencoding() encoded string with html entitys for non-defaultencoding characters. That is what Myghty did:
with regards to the patch, why would we go through all that trouble to parse the magic encoding comment when we are receiving a CompileException or SyntaxException ? in both cases, we have already lexed the template and we have already converted the source to unicode. this encoding could be passed along to CompileException and SyntaxException. also "parser" is not defined, and the whole BOM thing if we want to support that (which im not thrilled about, but I can be swayed) should just be part of Lexer.
the second part of the question, putting encoding_errors as a parameter to Template/TemplateLookup, i would say its not a big deal, the only reason its not there is because there already is a way to convert to unicode explicitly (more than one way to do it...but not a big deal)
ok so in [changeset:259] im propigating the source of the original template as a unicode object to the LexerException/CompileException, also ModuleInfo can get at the template source and its encoding from the generated module so also returns source as a unicode object. added tests for every version of this issue i could think of (memory based, file based, etc).