Does not handle object of objects

[joemastersemison]

I have a large (~2GB) JSON file that is a backup of a Firebase database. It is arranged as an object of objects (e.g.: {"objectA": {}, "objectB": [{},{}], "objectC": {}} ). When I import with pgfutter, I get "0 rows imported", even though pgfutter does seem to go through the whole file fairly quickly (less than 2 minutes).

I have experimented with trying to split the file with jq and writing my own stream-parsing code, but none of it seems to give me anything I can actually import properly with pgfutter. Can you help please?

[lukasmartinelli]

I would recommend using jq to split the object into several objects because you want to do that anyway if you want to bring your data into a good format. And it might be easier to do in jq than in SQL with the JSON operators.

Nontheless I implemented importing JSON objects in the latest v0.4 version. Check out whether that works for you.

https://github.com/lukasmartinelli/pgfutter#import-single-json-object
I do load the entire object into memory (I know kinda bad - but perhaps that still works for you).
The SQL API does not let me work with bytes directly, I need to pass the value when commiting the transaction I cannot work with streams - from what I know now.

[lukasmartinelli]

This feature has been released in v0.4.

pgfutter jsonobj document.json

[buildbreakdo]

What does this command do? There is no CLI output or database record..

pgfutter jsonobj document.json

[lukasmartinelli]

It should create a database record.