talking to Schema Registry needs local storage

[SMR39]

hi @dasch ,

I don't understand why do we need to maintain local copies of the schema when we can fetch them from the schema registry. Isn't the whole point of the schema registry is to maintain the latest copy of schema across all applications trying to produce and consume?

Encoding or decoding with avro_turf requires the schema files to be present at ./schemas directory (DEFAULT_SCHEMAS_PATH = "./schemas").
how can I just leverage the schemas present in the schema registry without storing them locally?

I also encounter this error when I try to run

require 'avro_turf/messaging'
require 'ruby-kafka'
require 'pry'

kafka = Kafka.new("kafka-0.x.net:32400,kafka-1.x.net:32401,kafka-2.x.net:32402".split(","))
avro = AvroTurf::Messaging.new(registry_url: "https://schema-registry.x.net/")

producer = kafka.producer

data = avro.encode({"org" => "foobar", "job_id" => "123abc"}, schema_name: "Source")
producer.produce(data, topic: "samples")
producer.deliver_messages

Traceback (most recent call last):
	2: from play.rb:14:in `<main>'
	1: from /Users/shashiravula/.rvm/gems/ruby-2.5.0/gems/avro_turf-0.8.0/lib/avro_turf/messaging.rb:52:in `encode'
/Users/shashiravula/.rvm/gems/ruby-2.5.0/gems/avro_turf-0.8.0/lib/avro_turf/schema_store.rb:23:in `find': expected schema `./schemas/Source.avsc' to define type `Source' (AvroTurf::SchemaError)

but then again if I change the schema name to namespace.schemaname. I can register the schema and decode the message using the same.

 pry(main)> sample = avro.encode(obj, schema_name: "betterdoctor.Source")
I, [2018-05-01T15:33:05.213874 #21628]  INFO -- : Registered schema for subject `betterdoctor.Source`; id = 2 

But outside of pry if use it in the example code above

data = avro.encode({"org" => "foobar", "job_id" => "123abc"}, schema_name: "betterdoctor.Source")
could not find Avro schema at `./schemas/betterdoctor/Source.avsc' (AvroTurf::SchemaNotFoundError)

I would really appreciate if you can help me tell where I am going wrong.

Best,
shashi

[dasch]

The difference in namespace usage looks like a problem with detecting your current working directory. Can you paste your schema definition?

Avro is designed around the notion that the source-of-truth schema is bundled with the code that writes data. The schema registry is a way to communicate the schemas to readers. It's possible to do things differently, e.g. by having a Rake task pull down the schemas from the registry before deploying your application, but I don't really see the value in it.

[SMR39]

Thanks for the Help @dasch. you were right, the namespace usage fixed it.

Best,
shashi