Dangerous network data parsing using eval()

[RemiCardona]

Using eval() to parse data received from the network is a huge security hole. There are 2 ways to fix this:

• the ast.literal_eval() function should happily parse Solr's pythonic output,
• use the json module on all python versions since mysolr seems to support python 2.6 and up.

I have opted for the second solution since the code being already there, I assumed it worked properly on newer python versions.

NB: I have tested this patch on python 2.6 against solr 3.6, on Debian Squeeze.

[moliware]

Hi,

Thanks for contributing! We started to use eval because evaluating a python expression is much faster than parsing a json or xml.

It's true that is less secure but we gain a lot of speed, so I think I will use your first way that it is also safer than using eval directly.

Thanks!!

[RemiCardona]

Hi Miguel,

I just did a couple of tests on a 5000-row resultset coming from solr. A friend of mine suggested I add anyjson to the list. Here's what I got:

• eval(): ~0.5s
• ast.literal_eval(): ~1s
• json.loads(): ~1.2s
• anyjson: ~0.1s

That last test just blew me off, so I triple-checked and anyjson is just a simple wrapper over various json libraries and it used simplejson on my test machine.

So I would definitely suggest using anyjson (which is available on pypi) or simplejson directly using a simple try/except ImportError.

Cheers

[moliware]

Wow, I'm going to test it and include anyjson.

Thank you so much for your contribution

[RemiCardona]

The updated patch works fine on my system. Cheers.

[moliware]

Hi,

From anyjson doc:

"Anyjson loads whichever is the fastest JSON module"

So, I will try each json library for seeing what library is the fastest in order to include this in the documentation. Then I will merge the pull request.

Thanks!