Compatibility issues with ubjson format

[jayendra13]

When I try to save the array with option ArrayToStruct, the size attribute doesn't get recovered. Here is a example where I creating ubjson out of matlab structure and dumping it to file.

>> d.a = randn(3,3);
>> d.b = 'hello';
>> opt.ArrayToStruct = 1;
>> opt.Compact = 1;
>> s = saveubjson('',d,opt);
>> fid = fopen('/tmp/ubjson.json','wb');
>> fwrite(fid,s);
>> fclose(fid);

Here in python I am opening the file and trying to reconstruct the structure of matlab, everything works well except the ArraySize attribute. shouldn't it be list like ArrayData?

In [92]: f = open('/tmp/ubjson.json','rb')

In [93]: data = f.read()

In [94]: f.close()

In [95]: d = ubjson.loadb(data)

In [96]: d
Out[96]: 
{u'a': {u'_ArrayData_': [5.235096335266677e+97,
   4.82949938982368e+217,
   -1.6371385346751378e+252,
   -2.173139193905037e-245,
   -1.050486479578402e+294,
   3.1118306554669778e+237,
   5.3658739553167216e+110,
   -6.843893294653657e-14,
   -1.1042601089134405e+282],
  u'_ArraySize_': '\x03\x03',
  u'_ArrayType_': u'double'},
 u'b': u'hello'}

[fangq]

@jayendra13, I apologize for the delay. I was out of town for a conference last week and did not get chance to write a detailed reply.

This compatibility issue is known. As described in Issue #6 and this README section

https://github.com/fangq/jsonlab/blob/master/README.txt#L366-L368

jsonlab utilizes an unofficial N-D array syntax to store complex arrays. This was originally proposed in this ubjson mailing list, but the upstream author of the specs had no interest to adopt

https://groups.google.com/forum/#!msg/universal-binary-json/tgMCEbOmhes/s7JlCl58hvQJ

Basically, in this extended syntax, the array dimension grammar (#) in UBJSON can be followed by an array object [[] ... []]. If this is detected, jsonlab will treat it as N-D array and reshape the 1D data stream into N-D based on the specified dimensions.

I felt that this extended syntax is extremely convenient when storing N-D arrays especially in matlab. This syntax also allows one to pre-allocate the array before reading an N-D array. If one has to do it with UBJSON Draft 9-12, he/she has to encapsulate each inner loop segment of an array using embedded array grammar. This is very inconvenient and hard to process.

I don't have a time table to fix the above Issue #6 because, in my opinion, the unofficial N-D array syntax in jsonlab is way more convenient for arrays of 2D or higher dimensions.

[fangq]

Duplicate of #6