DataFrame to_json Column -> [values] orientation and scientifc notation?

[MatthiasKauer]

Hi,
I'm exporting a data frame from pandas and importing it in a C++ program using rapidjson. I believe I have tested all available orientations and I am wondering if it was possible to obtain the following:

       c1 c2
    r1 1 2
    r2 3 4

converted to

{ c1: [1, 3], c2: [2,4], index: [r1, r2]}

Is my idea inherently flawed somehow? I feel it would be very intuitive to parse this from C++.

The other question is regarding the precision of the double values. I am only getting fixed-point values in the text output.
Can I get 1.234e-5 instead of 0.00001234 somehow?

If no, can you point out the process for possible contributions in this direction?

Best regards,
Matthias Kauer

[jreback]

cc @Komnomnomnom

[immerrr]

It is at least doable by

In [28]: df = pd.DataFrame(np.arange(1, 5).reshape(2,2), index=['r1', 'r2'], columns=['c1','c2'])

In [29]: df
Out[29]: 
    c1  c2
r1   1   2
r2   3   4

In [30]: df.reset_index().to_dict('list')
Out[30]: {'c1': [1, 3], 'c2': [2, 4], 'index': ['r1', 'r2']}

and then applying json.dumps separately.

[Komnomnomnom]

Hey @MatthiasKauer I'm afraid the answer is currently no to both your questions. Getting the desired output would require a new orient or using a third party encoder as @immerrr suggested. I'm not averse to adding a new orient, but if you want to take a look the initial changes would need to be made in objToJSON.c. It'd help if you're familiar with the numpy C-API and the JSON (ujson) code isn't the easiest to follow as there's a fair bit of jumping around and polymorphism going on.

Turning on/off scientific notation isn't supported and is unlikely to be (by me at least), is that just for human readability reasons?

[MatthiasKauer]

Thank you for the clarification. I didn't realize it was implemented with so much focus on performance.

The scientific notation was with regards to precision mostly. For my data, I often have to increase the number of digits.
In any case, neither of these things are too critical. Thank you for the help!