Support for registering schemas under custom subject

[arttsu]

As I understand, automatic schema registration always registers schemas under subjects named after the schema's full name (namespace + schema name). However, I've encountered a need to register a schema under a subject named after a kafka topic, because of the expectations of the system we're integrating with. I think this might be a pretty common usecase, as Confluent Schema Registry specifies TopicNameStrategy as the default https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#subject-name-strategy.

I was able to implement a workaround in our code by manually registering the schema

  defp ensure_registered_schema() do
    with {:ok, nil} <- Avrora.Storage.Memory.get(@schema_full_name) do
      {:ok, schema} = Avrora.Storage.File.get(@schema_full_name)
      {:ok, registered_schema} = Avrora.Storage.Registry.put("#{@topic_name}-value", schema.json)
      {:ok, _} = Avrora.Storage.Memory.put(@schema_full_name, registered_schema)
    end
  end

but I was wondering whether there's a simpler way to do it.

avro_turf allows specifying the subject name when encoding a payload https://github.com/dasch/avro_turf/blob/master/lib/avro_turf/messaging.rb#L87. It would be very convenient if Avrora would support something like that.

PS: Thanks for your work on the library! :)

[Strech]

Hey, @arttsu thanks for rising this question. In short, I think a better manual schema registration interface definitely a thing to implement, and I would mark it as an enhancement issue.

In specific auto-registration is:

1. Not a best-practice, but more a comprehensive feature when its conventions work for you
2. Many other features require to disable it since by design reader should never register the schema, it's a writer's prerogative
3. AvroTurf is good with some initial ideas, but it's not a good design solution to combine encoding interface with a schema registration, I would rather split the functionalities to follow SRP

WDYT, sounds good?

[arttsu]

@Strech, thanks for a thorough response! Glad that better manual schema registration would be on your radar. It would definitely be sufficient for my usecase.

[Strech]

Just a heads-up about schema registration, this is a sneak-peek for the upcoming functionality

I think it will not be exposed via the main Avrora module due to its specificity (I prefer to keep in Avrora only main flows about encoding/decoding), but can be used directly.

With it, your example can be rewritten as:

defp ensure_registered_schema() do
  {:ok, _} = Avrora.Utils.Registrator.register_schema_by_name(@schema_full_name, as: "#{@topic_name}-value")
end

[arttsu]

Thanks for working on this, looks great! Very minor nitpick - have you considered using the name Registrar for the new module? To me it seems to be more correct (not a native speaker though, so might be wrong 😄).

[Strech]

@arttsu thanks for the name suggestion, I will definitely look for the better name 👍🏼

[Strech]

@arttsu Thanks for your help, the #63 is in the master and will be released this week 🤝

[Strech]

@arttsu It takes me a while, but here we are https://github.com/Strech/avrora/releases/tag/v0.16.0 it includes also an updated mix task for schema registrations. I will prepare release notes soon. Thanks for your support 🤝

[arttsu]

Awesome! Thanks again for implementing and releasing this so quickly 👍