Investigate encode-for-tt to encode-for-pre switch... #1

kingcons opened this Issue Oct 15, 2012 · 5 comments


None yet
2 participants

kingcons commented Oct 15, 2012

From #lisp:
What's up?
Fade and I have an IRC bot implemented in Common Lisp which uses the colorize library to serve up source code files on a Web interface.
Oh nice. :)
The call we use is (colorize::colorize-file-to-stream :common-lisp source-file out-stream)
Sometime after your August updates, though, we started getting weird output. It's no longer breaking lines where there are line breaks in the source file, and we're also not seeing the indentation in source lines.
In the function html-colorization, there's a call to html-encode:encode-for-pre. It used to call encode-for-tt instead.
When I switch back to encode-for-tt, we get back our indentation and our line breaks.

Colorize continues to work fine for coleslaw but it's only being used under the covers by 3bmd, particularly the print-tagged-element method which in turn calls html-colorization.

What happened here? This is the commit in lisppaste that changed the call. It appears from the 3bmd source that the string resulting from html-colorization simply needs to be wrapped in a <PRE> block.


kingcons commented Oct 15, 2012

<3 quicklisp

(remove-if-not (lambda (x) (member "colorize" (ql-dist:required-systems x) :test #'equal))
               (ql:provided-systems t))

The above informs me that 3bmd, docbrowser, and sphinx are the only systems depending on colorize. Docbrowser only uses the scan-string function. Sphinx uses html-colorization for its visit-node method but would be easy to patch. It is likely in the best interest of our users to change html-colorization to return a code block already enclosed by <pre>.

Hi! As I am one now :) may I suggest the user be given a way to choose this (and a related behaviour)? I've just started using colorize in conjunction with javascript - POSTing colorization requests wrapped in FormData objects via XMLHttpRequest. For my use case it'd be better not to have pre tags added, although it doesn't matter much. The html-encode:encode-for-pre[tt] call itself is my main concern: I've found I can get elisp and common lisp '=' and '/' forms to be colorized and linkified just by adding the relevant characters to the colorize::symbol-characters string. But to do the same for '<' and '>' forms I also had to remove the call to html-encode:encode-for-pre. BTW, if I can find the time I'll have a go at making an up to date version of the elisp-symbols.lisp-expr file.


kingcons commented Nov 25, 2012

@p-l-h Good point. I'll get that done before the next ql update. And patches welcome, of course! :)


kingcons commented Nov 28, 2012

Fixed in commit 572afce. If you wish to use encode-for-tt, just add it as an optional arg to html-colorize: (html-colorize 'lisp "(+ 1 2 3)" 'encode-for-tt) or a keyword arg to colorize-file-to-stream: (colorize-file-to-stream 'lisp "foo.lisp" t :encoder 'encode-for-tt).

@kingcons kingcons closed this Nov 28, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment