TypeError: Decimal(...) is not JSON serializable with use_decimal=True

[pope1ni]

Hi,

I seem to have run into a problem when attempting to encode decimal types. I am running multiple applications on multiple workers using uWSGI. This problem essentially boils down to the use of multiple sub-interpreters.

In uWSGI a work around for this problem is to use the --single-interpreter option but this prevents multiple applications per process.

The problem in simplejson, I believe, is due to the caching of the Decimal type in the speed-ups C extension. This leads to a problem when checking if we have a decimal because the cached type is the same across all sub-interpreters, but the Decimal object compared is different across each sub-interpreter, i.e. the cache type in the extension is the type from the first sub-interpreter that imported simplejson and thus for subsequent sub-interpreters effectively "Decimal != Decimal".

I originally thought this may have been a problem with uWSGI and was pointed to the same issue that existed in psycopg2 < 2.4.3. I have updated psycopg2 and ruled that out as the cause, and deleting the _speedups.so library to force simplejson to use its python-based encoder caused the problem to disappear.

See the following links for more information:

• https://groups.google.com/d/msg/webpy/BLHnpjuQzLg/xEUDNW-R6TUJ
• dvarrazzo/psycopg@dde4c0d

I'm not very familiar with Python's C API unfortunately, otherwise I would have tried to offer you a patch.

I hope this won't be too awkward to fix.

Cheers,

Nick

[pope1ni]

As a follow up, the workaround I am currently using is to use DjangoJSONEncoder to force the Decimal conversion to string to happen Python-side. Alternatively (for non-Django users) the following could be used to prevent comparison of Decimal types in the C extension while still being able to have the benefit of the speed-ups for everything else:

import decimal, simplejson
class DecimalJSONEncoder(simplejson.JSONEncoder):
    def default(self, o):
        if isinstance(o, decimal.Decimal):
            return str(o)
        return super(DecimalJSONEncoder, self).default(o)

data = decimal.Decimal('10.0000000000')
print simplejson.dumps(data, cls=DecimalJSONEncoder)

[etrepum]

I believe that v2.5.1 should resolve your issue, let me know if it doesn't. I haven't tested it in an actual sub-interpreter environment.

607a489

[pope1ni]

Just tested... Works wonderfully, thanks :)