publish Kafka Connectors to maven central

[ryannedolan]

Hello, please consider publishing artifacts for the PubSub Kafka Connectors to Maven Central, thanks!

[dpcollins-google]

Can you clarify which repo this refers to?

[ryannedolan]

@dpcollins-google sorry, I updated the description, thanks!

[dpcollins-google]

Hi Ryan, I don't have any fundamental objection to this, but also don't really see how it would be valuable. Usually, connectors are uploaded as JARs to your kafka cluster, not interacted with as libraries. Is there some deployment scheme for connectors that would be significantly easier if they were available on maven central?

[ryannedolan]

Manually uploading jars doesn't scale to large fleets. We bundle supported Connectors along with our Connect runtime.

Even if we did manually upload jars, or maybe built some automation to do so, it's hard to manage artifacts that don't have Maven coordinates. Currently we have manually downloaded the jars from GitHub and manually uploaded these to Artifactory, which is less than ideal. We'd like to treat these as normal libraries, as that's how we're using them.

In future we'd also like to depend on these artifacts for integration tests and as base-classes to extend from, e.g. to add custom logging and metrics.

[dpcollins-google]

Thank you for the context. We'll consider this as a feature request.

[ryannedolan]

Thanks! I should also say that, if possible, we'd prefer such artifacts not be fat jars. I understand that fat jars are necessary for the upload-to-connect use-case, but not for our compile-time dependency use-cases.