Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SR: Extract Avro schema with reflection #1051

Merged
merged 20 commits into from
Jul 13, 2023
Merged

Conversation

lovromazgon
Copy link
Member

Description

This PR adds a function that traverses a struct (normally structured data) and produces an Avro schema from it. A good chunk of the code was taken from the SDK. Next time we add a format it would be advisable to extract the common code.

This approach still contains the drawback that it could produce a different schema if the provided value contains a nil value (we have no type info in that case).

Fixes #1041

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@lovromazgon lovromazgon requested a review from a team as a code owner May 30, 2023 18:31
pkg/processor/schemaregistry/avro/schema.go Outdated Show resolved Hide resolved
pkg/processor/schemaregistry/avro/schema.go Outdated Show resolved Hide resolved
pkg/processor/schemaregistry/avro/schema.go Outdated Show resolved Hide resolved
- add map union resolver
- add traversal functions
- document code
- improve tests
@lovromazgon lovromazgon mentioned this pull request Jun 30, 2023
4 tasks
pkg/processor/schemaregistry/avro/reflect.go Outdated Show resolved Hide resolved
pkg/processor/schemaregistry/avro/reflect.go Outdated Show resolved Hide resolved
pkg/processor/schemaregistry/avro/reflect.go Outdated Show resolved Hide resolved
pkg/processor/schemaregistry/avro/schema.go Show resolved Hide resolved
pkg/processor/schemaregistry/avro/schema.go Show resolved Hide resolved
pkg/processor/schemaregistry/avro/reflect.go Outdated Show resolved Hide resolved
@lovromazgon lovromazgon requested a review from hariso July 12, 2023 13:36
Copy link
Contributor

@hariso hariso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lovromazgon lovromazgon merged commit 5367324 into main Jul 13, 2023
3 checks passed
@lovromazgon lovromazgon deleted the lovro/sr-avro-reflect branch July 13, 2023 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SR: Extract Avro schema from structured data
2 participants