Support Karapace as an alternative Kafka schema registry devservice

[knutwannheden]

Description

Karapace is a lean Python-based and Confluent-compatible Kafka schema registry. As such it would be very useful to be able to use this instead of Apicurio as a devservice, when the schema registry used in production is the Confluent implementation. The Apicurio devservice has the drawback that it isn't Confluent-compatible and requires different serializers and deserializers for Avro-based messages.

See also discussion on Zulip: https://quarkusio.zulipchat.com/#narrow/stream/187030-users/topic/.E2.9C.94.20Kafka.3A.20Apicurio.20for.20dev.20and.20Confluent.20for.20prod.20mode.3F

Implementation ideas

Since the current devservice config properties are very Apicurio-specific (with an apicurio name segment), the evolution to support a different schema registry implementation may not be quite as straight-forward. Should the property names possibly be deprecated in favor of vendor-agnostic properties?

[quarkus-bot]

/cc @alesj, @cescoffier, @ozangunalp

[quarkus-bot]

You added a link to a Zulip discussion, please make sure the description of the issue is comprehensive and doesn't require accessing Zulip.

This message is automatically generated by a bot.

[alesj]

Afaik, Apicurio schema registry was designed to be Confluent-compatible -- at least when I was working on it (as initial contributer).
Is that not the case anymore?
@EricWittmann ^ ?

[cescoffier]

It is still the case, but it's a different endpoint. At the moment we inject the 'plain' apicurio url and not the confluent compatible one. This is something we should do if we detect the confluent serde on the classpath.

[knutwannheden]

@alesj Will #24526 already allow to use Karapace as an alternative?

[Ladicek]

Hi @knutwannheden, I worked on #24526 a little too so I think I can answer. That PR does not allow using Karapace as an alternative, because that isn't necessary. It always uses Apicurio Registry as a dev service, because Apicurio Registry provides a Confluent Schema Registry compatibility endpoint. Further, if you use the extension for Confluent Schema Registry that #24526 introduces, the Confluent Schema Registry libraries will be automatically configured (via SmallRye Reactive Messaging configuration) to use the compatibility endpoint, so everything should work out of the box. I hope that will be enough for you.

[cescoffier]

I agree with @Ladicek. The need was to be able to use the Confluent Avro Serde. This has been fixed in #24526.

I would be in favor to close this issue.

[cescoffier]

Closing, see comments.

[alesj]

Not yet. But let me check this and get back to you.

…

On Wed, 27 Apr 2022 at 21:33, Knut Wannheden ***@***.***> wrote: @alesj <https://github.com/alesj> Will #24526 <#24526> already allow to use Karapace as an alternative? — Reply to this email directly, view it on GitHub <#24350 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACRA6DTXGVSMKK7QTE3SXTVHGI7XANCNFSM5Q32SYPQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>