Wordpress importer doesn't translate [code]

[asmeurer]

In my Wordpress block, I made good use of their code blocks, which are delimited like

[code]
code here
[/code]

or, to use a language

[code language="py"]
python_code_here
[/code]

(I think language can also be spelled just lang).

If the wordpress importer uses Markdown, it could just translate [code] to , `[code lang='x']` to x, and enable fenced code blocks.

[Kwpolska]

This is a wordpress.com-only thing, documented here: http://en.support.wordpress.com/code/posting-source-code/

[asmeurer]

I just did a simple regex replace of \[/?code( lang(uage)?="(.*)")?\] with ```\3. It would have been nice if the importer did this for me, though.

[asmeurer]

One issue with it was that wordpress replaced things like > with >. So I guess it maybe needs to be smarter than this, using some <pre> environment that calls directly to the syntax highlighter.

[Kwpolska]

We’d need a smarter parser, because:

• lang(uage) is not required, it can be just [code]
• there are other meta fields we do not support
• < > & replacements would break it

This requires some work and time, though. Someone might take care of this, one day…

[asmeurer]

lang(uage) is not required, it can be just [code]

That is handled by my regular expression

The other problem I found is that there are blank lines between each line in [code] blocks.

[asmeurer]

There is already this https://github.com/getnikola/nikola/blob/master/nikola/plugins/command/import_wordpress.py#L352, but I don't understand what it is doing. What does the ~~~~~~~~~~~ do?

[Kwpolska]

probably some other form of fenced code blocks in python-markdown. (it’s also seemingly for another code-block extension. wordpress is a fucking mess.)

[asmeurer]

I am willing to take a shot at fixing this, since I have to do it anyway for my imported blog.

[Kwpolska]

Understood. Go on, and have fun. (make sure to read the documentation for the wordpress feature, linked above.)

[asmeurer]

I'll probably end up leaving the configuration parameters unimplemented, as I didn't use them.

[Kwpolska]

s/unimplemented/ignored/

s/I didn't/Nikola doesn’t/

[asmeurer]

The extra newlines are only appearing in my oldest blog posts. So probably that was some Wordpress thing that was fixed. There's no reliable way to detect the issue programmatically (unless we have some date that we know it changed).

[asmeurer]

OK, so I guess the only question I have is, what is the best way to convert things like < back to <?

[asmeurer]

s/unimplemented/ignored/

I was actually planning on unimplemented, but I guess I can do ignored too. I just need to make my regex a little more general.

[Kwpolska]

Just happily ignore the newline issue, as users are meant to review the output anyways.

For converting, you’d need to know that you are in a [code] block and do replacements there. But that is not easy, not without some hardcore parsing. (though I can think of a regexp cheat that goes like [code(.*?)](.*?)<(.*?)[/code] → [code\1]\2<\3[/code] — though this might very well fail, especially in corner cases, like ]<[ or ]aa<[ or ]<aa[.

[asmeurer]

I mean, is there a &whatever; to the appropriate actual character mapping somewhere (standard library, external library, already in Nikola, ...)?

[Kwpolska]

There are three you need to take care of:

& = &
&lt;  = <
&gt;  = >

[asmeurer]

Ah, the rest are just literal? I found http://www.ascii.cl/htmlcodes.htm. I think I also need ".

[asmeurer]

#1187