Comments from another independent SQLite JSONB implementation (in Python)

[ddevienne]

FIY, I announced your new support for SQLite JSONB to the SQLite forum.
I must be the only one (or close to) that is at the intersection of your library and SQLite :).
And got comments from Roger Binns: https://sqlite.org/forum/forumpost/a1bd0a4913, who also independently implemented that format.
FWIW.

[stephenberry]

Fantastic feedback! I'll carefully address this, thanks.

[rogerbinns]

Unfortunately you end up in the position of having to implement it as specified or as implemented, as they diverge. I saw you have fuzz testing - did you also try the randomjson extension?

[stephenberry]

Have you raised the issues of the divergence? I would think following the implementation would be best since this targets SQLite.

That randomjson fuzz testing looks very useful, thanks for pointing it out.

[rogerbinns]

Yes I did raise the divergences. One bug was fixed in the code (\v) and that was it. You'll note the spec says "version 3.45.0 (pending)" which came out in January 2024 and there have been 23 releases since, without any spec changes.

I tried really hard to get inf and nan to be handled well, but failed. (I wanted them preserved in JSON5 float).

SQLite only care about the JSONB and JSON conversions they produce, which is a subset of the spec. Also note that SQLite never validates UTF8 in any scenario or API. I do validate in my code because it is one way of telling the difference between a random bag of bytes and actual JSONB.

My opinion is that it should be as strongly specified and tested as the file format since it is effectively part of it now.