Recursive locking in Nokogiri::CSS::Parser.[]

[yorickpeterse]

When running our scraper setup under Rubinius I started to notice a handful of ThreadError errors with the message "Recursively locking not allowed". This particular error struck me as odd since Nokogiri objects aren't shared between threads in our code (to actually prevent any potential threading issues).

Looking at the stack trace it looks like the problem is caused by https://github.com/sparklemotion/nokogiri/blob/master/lib/nokogiri/css/parser_extras.rb#L19. Is there a particular reason for data being cached on class level? This looks like a rather big bottleneck and based on the errors we've been getting it's also actually becoming a problem. Having said that, luckily the amount of these errors so far has been very small, only 9 instances with about 20 minutes in between.

Sadly at this point I don't really have a way to reproduce it as I'm not even sure what exactly is causing the recursive lock. I'll do some extra investigating in the coming days.

[yorickpeterse]

To clarify: I've never seen this error pop up under MRI 1.9.3 in the past few months, most likely due to the GIL.

[yorickpeterse]

Part of the stacktrace (after the very last frame from our own code):

File "kernel/common/mutex.rb" line 38 in lock
File "kernel/common/mutex.rb" line 60 in synchronize
File "/var/www/review_collector/deploy-2014-01-24_16_22_05/vendor/bundle/rbx/2.1/gems/nokogiri-1.6.1/lib/nokogiri/css/parser_extras.rb" line 18 in []
File "/var/www/review_collector/deploy-2014-01-24_16_22_05/vendor/bundle/rbx/2.1/gems/nokogiri-1.6.1/lib/nokogiri/css/parser_extras.rb" line 72 in xpath_for
File "/var/www/review_collector/deploy-2014-01-24_16_22_05/vendor/bundle/rbx/2.1/gems/nokogiri-1.6.1/lib/nokogiri/css.rb" line 23 in xpath_for
File "/var/www/review_collector/deploy-2014-01-24_16_22_05/vendor/bundle/rbx/2.1/gems/nokogiri-1.6.1/lib/nokogiri/xml/node.rb" line 109 in search
File "kernel/bootstrap/array.rb" line 87 in map
File "/var/www/review_collector/deploy-2014-01-24_16_22_05/vendor/bundle/rbx/2.1/gems/nokogiri-1.6.1/lib/nokogiri/xml/node.rb" line 104 in search
File "/var/www/review_collector/deploy-2014-01-24_16_22_05/vendor/bundle/rbx/2.1/gems/nokogiri-1.6.1/lib/nokogiri/xml/node.rb" line 228 in at

[yorickpeterse]

Rubinius version is rubinius 2.2.3 (2.1.0 4792e746 2013-12-29 JI) [x86_64-linux-gnu]

[yorickpeterse]

Searching through our errors there are a few other locking errors going on, though it's not entirely clear if those are also related to Nokogiri (our error reporting app, Rollbar, apparently didn't log these).

The errors only occured today between 16:50 UTC and 18:30 UTC and only in about 5% of the jobs we processed. This makes me wonder how hard it will be to accurately reproduce these problems :/

[yorickpeterse]

This turned out to be an issue with Rubinius which has since been fixed (as far as I can tell).