New JSON-type fuzzing proposal

[nikitamikhaylov]

Company or project name

ClickHouse inc.

Use case

Since the new type has been merged #66444 we need to test it continuously.

Describe the solution you'd like

The proposal is relatively simple:

• CREATE TABLE (j JSON) ENGINE = X, X can be Memory, ReplacingMergeTree, SummingMergeTree, etc.
• Generate random JSON objects and generate INSERT statements based on them. We can try to use libprotobuf-mutator from LLVM and convert Protobuf to JSON or do something even smarter. fuzzJSON table function can also help here.

As a result we will have a bunch of JSON objects like {"a": 1, "b": 2} or {"c" : {"d": 3}} . Additionally we will track all the keys and how they nest to each other (in a form of a tree).

graph TD;
    j;
    j;
    j-->a;
    j-->b;
    j-->c;
    c-->d;

Loading

This diagram means in table j we have the following columns and subcolumns: j.a, j.b, j.c.d. We then can write down all possible paths and use them as identifiers for other queries like SELECTs, UPDATEs or DELETEs. They can be generated special mode of AST Fuzzer embedded into clickhouse-client.

We will pack everything together and then run in the CI.

@Avogar thoughts?

[alexey-milovidov]

Also, let's mine JSONs from the Internet and insert all of them.

For example, here are JSONs from GitHub comments: https://pastila.nl/?01209a23/2c182cb64f961702bd1ade1196cf4358#azk1p9WUcjjxhsEE2SbcuQ==

https://play.clickhouse.com/play?user=play#U0VMRUNUIGV4dHJhY3QoYm9keSwgJ3suK30nKSBBUyBqc29uIEZST00gZ2l0aHViX2V2ZW50cyBXSEVSRSBib2R5IExJS0UgJyV7IiV9JScgQU5EIGlzVmFsaWRKU09OKGpzb24pIExJTUlUIDEwMDA=