Protobuf support without schema registry.

[jlisam]

👋, we are evaluating ksqldb for an internal project but we ran into a roadblock. According to #4469, Protobuf support has been enabled; however it seems like the only way to use it is if you also use the schema registry (SR).

ksql> create table test with (kafka_topic='test-topic, value_format='protobuf');

Schema registry fetch for topic value request failed. Topic: test-topic
Caused by: KSQL is not configured to use a schema registry. To enable it, please
	set ksql.schema.registry.url

Unfortunately we do not use SR and it seems like there is no way to provide a pluggable serde when creating tables/streams. Any thoughts 😄 or workarounds ?

[jzaralim]

Hi @jlisam! It's true, SR is required for protobuf. Support for custom serdes has been brought up a couple of times (#1057) but it's not something we are working on yet.

At the top of my head, a possible workaround could be to use Kafka Streams to create a new topic in a different format.

[millerjp]

@jlisam you may want to look at this somewhat related issue also #6911

[jlisam]

@jzaralim thank you for the clarification. Hopefully you will support it in the future 😄 as ksql seems to be a great product