Qt console doesn't work with pygments 2.0rc1

[ccordoba12]

We have received two reports about this in Spyder already :-) This is the one I'll maintain open until it's fixed here:

https://code.google.com/p/spyderlib/issues/detail?id=2042

I'll see if I can solve this one given that you're busy with 3.0

[takluyver]

Oh dear, it looks like we're monkeypatching pygments. Let's try to eliminate said monkeypatch if it's at all possible.

[ccordoba12]

Where is that monkeypatch?

[takluyver]

https://github.com/ipython/ipython/blob/master/IPython/qt/console/pygments_highlighter.py#L67

[ccordoba12]

Removing the patch works. I'll investigate what other effects this change has.

[takluyver]

Thanks! I'm reading code trying to work out why it was monkeypatched in the first place.

[ccordoba12]

Ok, thanks!

[ccordoba12]

After reading pygments sources, it seems that changing this if/else block:

https://github.com/ipython/ipython/blob/master/IPython/qt/console/pygments_highlighter.py#L27

to

if action is not None:
    if type(action) is _TokenType:
        yield pos, action, m.group()
    else:
        for item in action(self, m):
            yield item

solves the problem without further efforts ;-)

[takluyver]

Hmmm. I'd really like to get rid of the monkeypatch properly, rather than just updating it to work with pygments at the moment.

[ccordoba12]

I understand. It's your call then.

[takluyver]

I have at least discovered what it's there for - highlighting inside multi-line strings goes wrong if I remove the monkeypatch. Now to find a better way to do it.

[takluyver]

I couldn't think of a good way to do this, so #6878 keeps the monkeypatch alive, as @ccordoba12 suggested.