Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation required for avro schema before creating it in schema registry #215

Open
sandeepk6 opened this issue May 11, 2023 · 1 comment
Labels
✨ Feature Request New feature or request

Comments

@sandeepk6
Copy link

schemaRegistry.createSchema proceeds and adds schema into schema registry without schema validation.
Creation is successful, but the writing of record to Kafka fails with an exception which is not useful to debug the issue.

Exception Trace:
ERRO[0001] panic: runtime error: invalid memory address or nil pointer dereference
goroutine 53 [running]:
runtime/debug.Stack()
scenarioruntime/debug/stack.go:24 +0x65
go.k6.io/k6/js/common.RunWithPanicCatching.func1()
go.k6.io/k6@v0.41.0/js/common/util.go:82 +0x191
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*Runtime).runWrapped.func1()
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2409 +0xdc
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*vm).try.func1()
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:626 +0x70f
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*Runtime).runWrapped.func1()
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2409 +0xdc
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*vm).try.func1()
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:626 +0x70f
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/linkedin/goavro/v2.(*Codec).NativeFromTextual(0xc00177ef50?, {0xc0036ea000, 0x186, 0x1a0})
github.com/linkedin/goavro/v2@v2.12.0/codec.go:500 +0x23
github.com/mostafa/xk6-kafka.(*AvroSerde).Serialize(0x18b3880, {0x18b4c60?, 0xc001f2de90?}, 0x4?)
github.com/mostafa/xk6-kafka@v0.16.0/avro.go:14 +0x79
github.com/mostafa/xk6-kafka.(*Kafka).serialize(0xc002558dc0, 0xc001f2de30)
github.com/mostafa/xk6-kafka@v0.16.0/serdes.go:46 +0x119
github.com/mostafa/xk6-kafka.(*Kafka).schemaRegistryClientClass.func4({{0x1dd64d8, 0xc0021546f0}, {0xc0025fe8a0, 0x1, 0x1a}})
github.com/mostafa/xk6-kafka@v0.16.0/schema_registry.go:210 +0x12d
github.com/dop251/goja.(*vm)._nativeCall(0xc002537300, 0xc0000bb900, 0x1)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:3233 +0x1e3
github.com/dop251/goja.call.exec(0xa?, 0xc002537300)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:3203 +0x33b
github.com/dop251/goja.(*vm).run(0xc002537300)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:495 +0x8b
github.com/dop251/goja.(*baseJsFuncObject)._call(0xc0029a3d40, {0x0?, 0x0, 0x1e61b060598?}, {0x0?, 0x0}, {0x1dd6dc0?, 0x2950a20?})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/func.go:318 +0x3f8
github.com/dop251/goja.(*baseJsFuncObject).call(...)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/func.go:328
github.com/dop251/goja.(*baseJsFuncObject).Call(0xaa0853?, {{0x1dd6dc0, 0x2950a20}, {0x0, 0x0, 0x0}})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/func.go:280 +0xa5
github.com/dop251/goja.AssertFunction.func1.1()
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2369 +0x74
github.com/dop251/goja.(*vm).try(0xc002537300, 0x110091b?)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:634 +0x18a
github.com/dop251/goja.(*Runtime).runWrapped(0xc0027a3880, 0xc000499000?)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2413 +0x7c
github.com/dop251/goja.AssertFunction.func1({0x1dd6dc0?, 0x2950a20?}, {0x0?, 0x4?, 0xc001522c30?})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2368 +0x92
go.k6.io/k6/js/modules/k6.(*K6).Group(0xc0002fd9b0, {0xc00191c2e0, 0x1b}, 0xc0026ac978)
go.k6.io/k6@v0.41.0/js/modules/k6/k6.go:122 +0x203
reflect.Value.call({0x18aa100?, 0xc0002fd9e0?, 0x1e61b060598?}, {0x1a653ba, 0x4}, {0xc001d549c0, 0x2, 0xc0006af060?})
reflect/value.go:584 +0x8c5
reflect.Value.Call({0x18aa100?, 0xc0002fd9e0?, 0xc001d54930?}, {0xc001d549c0?, 0xc002622d10?, 0xc000088000?})
reflect/value.go:368 +0xbc
github.com/dop251/goja.(*Runtime).wrapReflectFunc.func1({{0x1dd6dc0, 0x2950a20}, {0xc0025fe7d0, 0x2, 0x27}})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:1952 +0x1da
github.com/dop251/goja.(*vm)._nativeCall(0xc002537300, 0xc0029a3c80, 0x2)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:3233 +0x1e3
github.com/dop251/goja.call.exec(0x5?, 0xc002537300)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:3205 +0x14d
github.com/dop251/goja.(*vm).run(0xc002537300)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:495 +0x8b
github.com/dop251/goja.(*baseJsFuncObject)._call(0xc002204300, {0xc000617c80?, 0x1, 0x8?}, {0x0?, 0x0}, {0x1dd6dc0?, 0x2950a20?})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/func.go:318 +0x3f8
github.com/dop251/goja.(*baseJsFuncObject).call(...)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/func.go:328
github.com/dop251/goja.(*baseJsFuncObject).Call(0x2030007?, {{0x1dd6dc0, 0x2950a20}, {0xc000617c80, 0x1, 0x1}})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/func.go:280 +0xa5
github.com/dop251/goja.AssertFunction.func1.1()
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2369 +0x74
github.com/dop251/goja.(*vm).try(0xc002537300, 0x0?)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/vm.go:634 +0x18a
github.com/dop251/goja.(*Runtime).runWrapped(0xc0027a3880, 0x8?)
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2413 +0x7c
github.com/dop251/goja.AssertFunction.func1({0x1dd6dc0?, 0x2950a20?}, {0xc000617c80?, 0x1?, 0x1?})
github.com/dop251/goja@v0.0.0-20221118162653-d4bf6fde1b86/runtime.go:2368 +0x92
go.k6.io/k6/js.(*VU).runFn.func2.1()
go.k6.io/k6@v0.41.0/js/runner.go:774 +0x43
go.k6.io/k6/js/eventloop.(*EventLoop).Start(0xc0007610e0, 0xc001d54780)
go.k6.io/k6@v0.41.0/js/eventloop/eventloop.go:171 +0x1b7
go.k6.io/k6/js.(*VU).runFn.func2()
go.k6.io/k6@v0.41.0/js/runner.go:773 +0xeb
go.k6.io/k6/js/common.RunWithPanicCatching({0x1dd9a60?, 0xc0007d7200?}, 0xa2f986?, 0x18d6de0?)
go.k6.io/k6@v0.41.0/js/common/util.go:86 +0x87
go.k6.io/k6/js.(*VU).runFn(0xc002491bc0, {0x1dc9be0, 0xc0025bed80}, 0xc0?, 0xc0001f3d58, 0xc000617c70, {0xc000617c80, 0x1, 0x1})
go.k6.io/k6@v0.41.0/js/runner.go:772 +0x28a
go.k6.io/k6/js.(*ActiveVU).RunOnce(0xc0025bed40)
go.k6.io/k6@v0.41.0/js/runner.go:717 +0x412
go.k6.io/k6/lib/executor.getIterationRunner.func1({0x1dc9c88, 0xc001d54510}, {0x1dbf760?, 0xc0025bed40?})
go.k6.io/k6@v0.41.0/lib/executor/helpers.go:124 +0x64
go.k6.io/k6/lib/executor.PerVUIterations.Run.func5({0x1dc51c0, 0xc002491bc0})
go.k6.io/k6@v0.41.0/lib/executor/per_vu_iterations.go:227 +0x40a
created by go.k6.io/k6/lib/executor.PerVUIterations.Run
go.k6.io/k6@v0.41.0/lib/executor/per_vu_iterations.go:240 +0xc29

Avro Schema that was created and was allowed:
"namespace": "test.1",
"type": "record",
"name": "myschema",

As per Avro official documentation, it looks like namespace cannot start with a number like 1.test etc. Avro schema validation is highly desired that should not allow an invalid schema to be added to schema registry.

@mostafa
Copy link
Owner

mostafa commented May 12, 2023

Hey @sandeepk6,

It is expected of the developer/tester to provide valid schema in the code, which is why there is no validation. But I'd be happy to review contributions for schema validation.

@mostafa mostafa added the ✨ Feature Request New feature or request label May 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ Feature Request New feature or request
Projects
Status: Todo
Development

No branches or pull requests

2 participants