import_logs.py doesn't support Unicode characters with json format

[estemendoza]

I'm working with iOS apps and sometimes their names have special characters and when this apps make requests to my server, the import_log.py script fails to parse this lines. This only happens when using nginx_json format.

Here's an example:

{"ip": "1.1.1.1","host": "blabla.com","path": "/api/test.xml","status": "200","referrer": "-","user_agent": "F\xFAtbol/1.0 (iPhone; iOS 7.1; Scale/2.00)","length": 267,"generation_time_milli": 0.009,"date": "2014-04-16T14:56:24-03:00"}

Check that user_agent has a special character and that the script fails when parsing this lines.

Thanks

Migrated from matomo-org/matomo#5013

[diosmosis]

This is a bug in nginx, \xFA is not valid JSON nor is it valid UTF-8. It is actually latin-1 for ú (so the user agent is Fútbol...). Unfortunately, treating individual JSON strings within serialized JSON as a separate encoding seems to be non-trivial, so I have yet to find a workaround I can apply to the log importer. (I tried using python-cjson which can handle hex escapes, but it has other differences w/ the normal json module.)

[diosmosis]

Found a workaround: #120 will merge momentarily.