Use turbo json blobs not inserting?

[CDBridger]

when we turned on use turbo one of the tables columns seemed to just come up blank. It was a column which stored some json blobs as strings. The funny thing is we have another table which has pretty much the same schema which is doesn't have the issue we are seeing.

Here is an example of data that we have the turbo on for the initial sync:

{
    "project_id": "4vh6vdb309sxua1t",
    "number": "4201",
    "registeredBy_id": "h7qwkegt5xma80e9",
    "registerDate": 1665715934621,
    "projectDayworks_id": "6jfxoyarzkxxp94o",
    "status": "draft",
    "floorplanReference_id": "qbfr9d3bsgk8pex4",
    "area_id": "c38q94r6c1bzez7g",
    "type": "penetration",
    "isVisible": true,
    "isLocked": false,
    "displayProps": "{\"geometry\":{\"type\":\"Point\",\"coordinates\":[882.9422402733347,1834.7897312388345]}}",
    "lowDisplayProps": "{\"geometry\":{\"type\":\"Point\",\"coordinates\":[882.9422402733347,1834.7897312388345]}}",
    "dataProps": "{\"type\":\"penetration\",\"isLocked\":false}",
    "quotePropType": "",
    "solution_id": "3ns0iif7gjbfgvbp",
    "solutionService_id": "islmcwi3jpc8ladl",
    "solutionSubstrate_id": "hto9nmbz4xkp0hhy",
    "ratingDimensionOne": 1,
    "ratingDimensionTwo": 30,
    "ratingDimensionThree": 30,
    "sizeCategoryName": "Circular",
    "notes": "",
    "name": "4201",
    "created_at": 1665715953585,
    "updated_at": 1668634362000,
    "id": "m9136l3ti7v1g7pq"
  } 

in the object above the property "dataProps" always gets inserted as null, for every entry in that table.

   {
    "floorplanReference_id": "i0phpxgsunn5frqv",
    "area_id": "yk20z0n3bmvdz8ty",
    "type": "area",
    "isVisible": true,
    "isLocked": false,
    "displayProps": "{\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[689.5989980128202,1325.908859896178],[988.2366828777481,1325.908859896178],[988.2366828777481,1293.7972808784439],[1827.2427572173688,1293.7972808784439],[1827.2427572173688,1196.9703284377397],[689.5989980128202,1196.9703284377397],[689.5989980128202,1325.908859896178]]]}}",
    "lowDisplayProps": "{\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[689.5989980128202,1325.908859896178],[988.2366828777481,1325.908859896178],[988.2366828777481,1293.7972808784439],[1827.2427572173688,1293.7972808784439],[1827.2427572173688,1196.9703284377397],[689.5989980128202,1196.9703284377397],[689.5989980128202,1325.908859896178]]]}}",
    "dataProps": "{\"area\":119744.30578217609,\"type\":\"area\",\"isLocked\":false,\"stroke\":{\"color\":\"rgba(230, 83, 230, 1)\"},\"fill\":{\"color\":\"rgba(230, 83, 230, 0.25)\"}}",
    "ratingDimensionOne": 0,
    "ratingDimensionTwo": 0,
    "ratingDimensionThree": 0,
    "name": "Passage",
    "created_at": 1669229723674,
    "updated_at": 1669229781510,
    "id": "ox7u4q2ky7hj1cgp"
  }

the object above is from another table and it works fine.

EDIT: I've extracted and formatted these objects for the sake of this post. when we use turbo it comes as a 70mb string.

[CDBridger]

I tried looking into the code and I can see that it uses v1.0.0 of simdjson, which is now at version 3.x. I briefly looked though the issues and saw that between 1.0.0 and the current there are some json parsing bugs that were fixed. I'm wondering if it is that? Is there any chance we have the dependency of simdjson updated?

[CDBridger]

Looking into it further I can see in the shared Database.cpp file, when setting a breakpoint, at this code

if (column.type == ColumnType::string && type == ondemand::json_type::string) {
   std::string_view stringView = value;
   sqlite3_bind_text(stmt, argumentsIdx, stringView.data(), (int) stringView.length(), SQLITE_STATIC);
 }

I can see the stringView has the data, but it size/capacity is 0 which is what is causing the bug I assume

[radex]

@CDBridger Hey, that's very strange, can you see if the latest 0.26 pre-release fixes the issue? I've updated simdjson to the latest version

[CDBridger]

cheers, I'll have a look and report back.

[CDBridger]

Looks like its solved, sorry it took so long, been very busy on multiple other projects.