Queries interface fails if response has non-utf8 symbols

[Krock21]

Queries interface fails if response has non-utf8 symbols

How to reproduce:

1. Submit "привет" CHYT query.
2. See how interface fails with an error Error occurred while parsing YSON[1], Invalid UTF-8 string in JSON[1]

Or use CLI to reproduce:

yt start-query chyt "привет"
queryID
yt get-query queryID --format '<encode_utf8=%false>json'
HTTP 400 Error occurred while parsing YSON Invalid UTF-8 string in JSON

This is because CHYT converts clickhouse-thrown std::exception into YT TError directly, in binary format (using exception.what())

Then it is stored in dynamic table in binary format:

"inner_errors" = [
        {
            "attributes" = {
                "datetime" = "2024-03-08T17:27:47.033681Z";
                "fid" = 0u;
                "host" = "our-host.net";
                "pid" = 3673;
                "thread" = "ThreadPool";
                "tid" = 7905737751553949491u
            };
            "code" = 1;
            "message" = 53 79 6e 74 61 78 20 65 72 72 6f 72 3a 20 66 61 69 6c 65 64 20 61 74 20 70 6f 73 69 74 69 6f 6e 20 31 20 28 27 d0 27 29 3a 20 d0 be d1 88 d0 b8 d0 b1 d0 ba d0 b0 31 2e 20 55 6e 72 65 63 6f 67 6e 69 7a 65 64 20 74 6f 6b 65 6e 3a 20 27 d0 27
        }

and can't be decoded into UTF8 json on server side, resulting in failure

[Krock21]

As a solution we can just use 'json' format. It should have no effect on other UI internals that parse json (unless they behave differently on unicode)

[Krock21]

I researched it a bit, and seems like current way of fetching data (format <encode_utf8=%false>json is correct

The issue is that binary yson written in table is not a correct UTF8 sequence, hence YSON -> JSON conversion fails

For this case, I think query-tracker/CHYT should make sure that correct utf8 is written. I will investigate further and come back with results

[Krock21]

After some discussion we decided that binary exceptions are okay and UI should learn how to handle them.

UI can handle binary data on Operation page, since it does get_operation format=json and decodes output to correct UTF8 or shows it in binary
UI can handle binary data from tables by using web_json format. web_json isn't available for non-streaming APIs so it is not a solution

A suggested solution for UI is to never use <encode_utf8=false>json since it fails on non-utf8 data. UI should use other format that is okay with binary data and perform additional decoding

[vrozaev]

Fixed in #533