MentosError header errors #45

aprescott opened this Issue Oct 18, 2012 · 33 comments


None yet
1.9.3p286 :001 > require "pygments"
 => true 
1.9.3p286 :002 > Pygments.highlight("foo")
MentosError: Failed to get header.
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:351:in `rescue in get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:332:in `get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:229:in `block in mentos'
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:203:in `mentos'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:186:in `highlight'
    from (irb):2
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'

As far as I can tell, this is happening on a clean gem environment.

Curiously, installing python-pygments and calling Pygmentize.start("pygmentize") first gives, for one single subsequent call to highlight a slightly different message:

1.9.3p286 :003 > Pygments.start("pygmentize"); Pygments.highlight("foo")
MentosError: No header received back.
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:295:in `handle_header_and_return'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:232:in `block in mentos'
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:203:in `mentos'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:186:in `highlight'
    from (irb):3
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'

1.9.3p286 :004 > Pygments.highlight("foo")
MentosError: Failed to get header.
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:351:in `rescue in get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:332:in `get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:229:in `block in mentos'
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:203:in `mentos'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:186:in `highlight'
    from (irb):4
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'

tnm commented Oct 20, 2012

Can you let me know what version of Python you are running on this machine?


python --version is 3.3.0.


I can confirm the same bug with Python 3.2.3, however it works fine with Python 2.7.3 (both on Gentoo Linux). Probably a Python 3 issue.


What's the best way for me to specify a different executable to pygments.rb in the meantime? I have python2 on my PATH.


(The README used to mention, e.g., RubyPython.configure :python_exe => 'python2.7', but it seems 0.3 has moved to a different IPC system.)

tnm commented Dec 7, 2012

We're officially only supporting 2.5, 2.6, and 2.7 for now. I'm planning support for Python 3, and patches are welcome for now.


Good to know, thanks. My ~/bin/python + PATH workaround is working for now.


Having the same issue here! python2 and python2.7 are in my path! What to do?

Ivoz commented Jan 21, 2013

pygments should not assume that the system default python is going to be v2.x and not v3.x

It should be easy enough to find if a python2 exists on the path, and use that instead, no? Given that v2.x is now pretty much at EOL.

obsoke commented Jan 24, 2013

I've been getting this error as well in Jekyll. Using pygments 0.3.7. I have 'python2' aliased to python & in my path.

Has anyone managed to resolve this?

edit: Editing lib/pygments/'s first line from 'python' to 'python2' seemed to fix it for me.


Has anyone managed to resolve this?

I just created ~/bin/python as a symlink to python2. ~/bin is in my PATH. I didn't need to edit anything in


I’ve got the same error, but I’m using Python 2.7. As far as I know I don’t have Python 3 installed on my machine at all (Mac 10.6, Python installed via Homebrew):

1.9.3-p385 :001 > require 'pygments'
 => true 
1.9.3-p385 :002 > Pygments.highlight 'foo'
MentosError: Failed to get header.
        from /Users/matt/.rvm/gems/ruby-1.9.3-p385/gems/pygments.rb-0.3.7/lib/pygments/popen.rb:357:in `rescue in get_header'
        from /Users/matt/.rvm/gems/ruby-1.9.3-p385/gems/pygments.rb-0.3.7/lib/pygments/popen.rb:338:in `get_header'
        from /Users/matt/.rvm/gems/ruby-1.9.3-p385/gems/pygments.rb-0.3.7/lib/pygments/popen.rb:235:in `block in mentos'
        from /Users/matt/.rvm/rubies/ruby-1.9.3-p385/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
        from /Users/matt/.rvm/gems/ruby-1.9.3-p385/gems/pygments.rb-0.3.7/lib/pygments/popen.rb:209:in `mentos'
        from /Users/matt/.rvm/gems/ruby-1.9.3-p385/gems/pygments.rb-0.3.7/lib/pygments/popen.rb:192:in `highlight'
        from (irb):2
        from /Users/matt/.rvm/rubies/ruby-1.9.3-p385/bin/irb:16:in `<main>'

In order to try and track the issue down I tried running the script independently. It failed with the error Illegal instruction, which seems to be produced by the line os.close(fd). Commenting out this line (actually the whole for block so the indentation is right) clears the error, and commenting the same block in the gem itself seems to fix it:

1.9.3-p385 :001 > require 'pygments'
 => true 
1.9.3-p385 :002 > Pygments.highlight 'foo'
 => "<div class=\"highlight\"><pre><span class=\"n\">foo</span>\n</pre></div>" 

Can anyone cast any light on this?

tnm commented Mar 28, 2013

@mattwildig Thanks for that. That can be fixed. I'll investigate.


@tnm It appears that this error has fixed itself. Whilst I could reliably reproduce it when I reported it, it is no longer appearing. Unfortunately I don’t know what changed between than and now that could affect this. I did restart my machine recently (after installing updates) so that might have cleared something in the OS.

ekampp commented May 28, 2013

I have this problem appear intermittent on Heroku, when attempting to parse some markdown. The problem seems to be spread across all OS, browsers and countries, and there is no discernible pattern. Here is an example error:

MentosError: File "/app/vendor/bundle/ruby/2.0.0/gems/pygments.rb-0.5.0/lib/pygments/", line 282, 
in start header = json.loads(line) File "/usr/local/lib/python2.7/json/", line 326, in loads return 
_default_decoder.decode(s) File "/usr/local/lib/python2.7/json/", line 369, in decode raise 
ValueError(errmsg("Extra data", s, end, len(s))) ValueError: Extra data: line 1 column 1 - line 1 column 129 
(char 1 - 129)
radum commented Jun 13, 2013

For Windows users, Python can run with multiple versions without any problem, so just install Python 2 and 3 and just change the env path. I know it is not really useful but could get you started at least.

tschaub commented Jun 25, 2013

@mattwildig - thanks for posting your fix here. I was getting the same with Ruby 1.8, Pygments 0.5.0, and Python 2.6.1. Commenting out the relevant block in fixed the issue for me (and now Jekyll works again - see jekyll/jekyll#1181).

milinda commented Jul 6, 2013

I got the same errror in Mountain Lion with Ruby 2.0.0p0 and Python 2.7.2. But it was not coming from the code block mentioned above. It was coming from ruby-2.0.0-p0/gems/pygments.rb-0.5.1/lib/pygments/popen.rb:366.

It says "in `rescue in get_header': Failed to get header. (MentosError)". Please let me know if any of you have any idea about this.

milinda commented Jul 6, 2013

In my case popen.rb's get_header method's size[0..-2] failed saying

undefined method `[]' for nil:NilClass

It looks like size = returns a null. This may be caused by a different error.

milinda commented Jul 9, 2013

Finally I found the cause and a solution for my issue. It was due to corrupted python installation on Mac OS X Mountain Lion. pygmentize was throwing errors due to a permission issue.


@milinda So wats the fix?


I ran into this too. This guy's fix worked temporarily, but it would be great if there was a real fix ;).

Ivoz commented Aug 30, 2013

You can also run the tool in a python virtualenv that uses python2, for a workaround.

cdgz commented Nov 22, 2013

Just to confirm that issue persists in 0.5.4 with ruby 1.8.7 and python 2.7.2 on SuSE Linux (3.0.13-0.27)

With python 2.6.7 Ok.

phansch commented Dec 7, 2013

I had the same error but it seems for me the cause was something different? Running gave me the following exception:

$ python
Traceback (most recent call last):
  File "", line 17, in <module>
    from pygments import lexers, formatters, styles, filters
  File "/home/philipp/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/pygments.rb-0.5.0/vendor/pygments-main/pygments/lexers/", line 18, in <module>
    from pygments.plugin import find_plugin_lexers
  File "/home/philipp/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/gems/2.0.0/gems/pygments.rb-0.5.0/vendor/pygments-main/pygments/", line 39, in <module>
    import pkg_resources
  File "/usr/lib/python2.7/dist-packages/", line 2823, in <module>
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python2.7/dist-packages/", line 710, in subscribe
  File "/usr/lib/python2.7/dist-packages/", line 2823, in <lambda>
    add_activation_listener(lambda dist: dist.activate())
  File "/usr/lib/python2.7/dist-packages/", line 2255, in activate
  File "/usr/lib/python2.7/dist-packages/", line 2362, in insert_on
  File "/usr/lib/python2.7/dist-packages/", line 2401, in check_version_conflict
    for modname in self._get_metadata('top_level.txt'):
  File "/usr/lib/python2.7/dist-packages/", line 2249, in _get_metadata
    for line in self.get_metadata_lines(name):
  File "/usr/lib/python2.7/dist-packages/", line 1219, in get_metadata_lines
    return yield_lines(self.get_metadata(name))
  File "/usr/lib/python2.7/dist-packages/", line 1211, in get_metadata
    return self._get(self._fn(self.egg_info,name))
  File "/usr/lib/python2.7/dist-packages/", line 1326, in _get
    stream = open(path, 'rb')
IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/oauthlib-0.6.0-py2.7.egg/EGG-INFO/top_level.txt'

Some time ago I installed a package via sudo easy_install. I don't use the python setuptools very often so maybe I did something wrong there.
For some reason insisted on using the oauth package in /usr/local/lib/python2.7/dist-packages/... I removed that (sudo rm -rf /usr/local/lib/python2.7/dist-packages/oauthlib-0.6.0-py2.7.egg) and now jekyll is running again. Now, this is just a workaround because that probably breaks the python package, right?

Ivoz commented Dec 7, 2013

I'm not sure without knowing why you have a python 2.7 in both /usr and /usr/local


(The README used to mention, e.g., RubyPython.configure :python_exe => 'python2.7', but it seems 0.3 has moved to a different IPC system.)

Actually, pygments already support python 3 now, but it should install with python install or other tool, it will use 2to3 to turn some 2 specified syntax into 3-compatible. (such as turning except Error, err to except Error as err ). The original source code is just python 2 compatible. (see here) in pygments.rb just modifies sys.path in order to import pygments. I think maybe pygments.rb should hack with, modify sys.path accroding to python executable version, for instance, add another python 3 compatible copy of pygments code

base_dir = dirname(dirname(dirname(os.path.abspath(__file__))))
sys.path.append(base_dir + "/vendor")
if sys.version_info[0] > 2:
    sys.path.append(base_dir + "/vendor/pygments-main3")
    sys.path.append(base_dir + "/vendor/pygments-main")
sys.path.append(base_dir + "/vendor/simplejson")

I fixed by re-install python 2.7.6. Refer to here:
jekyll/jekyll#1181 (comment)


This has reared its head again for me, so I’ve had another look.

I’m adding my comment here since it continues from my previous one in this thread but arguably could warrant its own issue, and doesn’t apply to everyone seeing this error (e.g. this isn’t related to the Python2/Python3 problem, except that the error is the same). The exception is raised when there is an issue communicating with Python and could result from a number of underlying causes.

My problem is the same as before, an Illegal instruction caused by the line os.close(fd) in Digging a bit further I came up with this Python script that reproduces the problem on my machine:

import os

# this is from
import pkg_resources

maxfd = 65536

# this is the loop from
for fd in range(3, maxfd):

The import pkg_resources is from within Pygments, and changing that try...except block to just do pkg_resources = None (i.e. as if the package wasn’t found) fixes the problem when calling from Ruby.

With a little more digging I can actually get the same issue with this smaller snippet:

import os, time

import pkg_resources

# note no try...except needed, this is actually closing an fd


What seems to be happening is Pygments is importing pkg_resources, which is opening a file or doing something else to get a file descriptor. is then closing that fd, assuming it has been inherited from Ruby. After it has been closed something happens in Pygments or pkg_resources that expects the fd to be open, and that causes the Illegal instruction. What that something is, what the file being opened is, why it always seems to be fd 4, whether it will always be fd 4 and why it seemed to be working for a while on my machine are all things I don’t know.

Is this something that can be fixed in pygments.rb? Looking at the code it seems it will only affect people with pkg_resources which might be why it hasn’t been a bigger problem. I should point out I’m still on Snow Leopard (I should get round to upgrading) – I don’t know if that would make a difference. Can anyone reproduce with my snippets on a later Mac version or different OS?

Ivoz commented Apr 28, 2014

you could use lsof while the script is open (do a sleep first, before the close) to check what file is being open that's causing the issue


Thanks @Ivoz. lsof produces this line which looks like the culprit:

python  41894 matt    4u  KQUEUE                                count=1, state=0x2
@kimpenhaus kimpenhaus referenced this issue in octopress/octopress Jan 21, 2015

octopress docs fails on windows #77


For the record, I had the same issue on OpenBSD with:

  • python 2.7.6
  • ruby 2.1.0
  • posix-spawn 0.3.9
  • pygments.rb 0.6.0


python /usr/local/lib/ruby/gems/2.1/gems/pygments.rb-0.5.0/lib/pygments/

was just fine (no output, waiting for some input).

The problem was with posix-spawn. Sorry but no more log about that.

gem install posix-spawn --version "=0.3.8"
gem uninstall posix-spawn --version ">0.3.8"

solved the issue.

frenkel commented Jan 25, 2015

Thanks @paulfariello I had the same problem on OpenBSD!

@vtamara vtamara referenced this issue in rtomayko/posix-spawn Jun 5, 2015

Does not compile on OpenBSD #24

@xupeng xupeng added a commit to xupeng/octopress-in-docker that referenced this issue Feb 26, 2016
@xupeng xupeng fix octopress "liquid error" 5eadbc3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment