Always read and insert Kafka messages as a whole

[abyss7]

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

For changelog. Remove if this is non-significant change.

Category (leave one):

• Improvement

Short description (up to few sentences):
Insert all rows from a single Kafka message atomically.

[filimonov]

https://clickhouse-test-reports.s3.yandex.net/6950/f35de2aafb65b8d4e5fc3d529635ed36da18c137/pvs_studio_report/pvs-studio-html-report/sources/BlocksBlockInputStream.h_14.html#ln26 relevant?

[abyss7]

https://clickhouse-test-reports.s3.yandex.net/6950/f35de2aafb65b8d4e5fc3d529635ed36da18c137/pvs_studio_report/pvs-studio-html-report/sources/BlocksBlockInputStream.h_14.html#ln26 relevant?

Yep, that's wrong. Anyway I will remove all unrelated changes including these ones.

[ghuname]

Hi,

in ClickHouse release 19.15.2.2, 2019-10-01 change log there is an improvement:
Every message read from Kafka is inserted atomically. This resolves almost all known issues with Kafka engine. #6950 (Ivan), which is this issue.

Can you please explain in few words how this atomicity in this context work?

Clickhouse used to consume batch of messages, and if a single message is wrong, the whole batch is lost. How it works now?

Regards.