Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support reader schema when decoding avro #166

Merged
merged 6 commits into from
Nov 22, 2021
Merged

Conversation

Nevon
Copy link
Member

@Nevon Nevon commented Nov 4, 2021

I can't push to invitae:add-avro-schema-resolution, so I'm opening up this as a continuation of #137

This changes the proposed interface slightly, as to not pollute non-Avro decoding with a readerSchema option that does nothing. It also tries to minimize the amount of types used directly from avsc and instead favors our own Schema types were possible. The reason for this is twofold:

  1. Reduce the risk that changes to avsc will affect our users. For example, if avsc released a new version where the Type type isn't compatible with the old version, users would be restricted to the version we depend on, even if the actual changes don't affect us.
  2. Ensure that you can read a schema using our utilities (readAvsc etc) and use those as reader schemas. This means that we internally turn them into avsc Type instances when they are used as reader schemas.

This means that the interface becomes:

let readerSchema = await readAvscAsync('/some/path')
await registry.decode(payload, {
  [SchemaType.AVRO]: {
    readerSchema
  }
})

// Similarly, you should be able to read a schema from the registry and use it as a reader schema
readerSchema = await registry.getSchema(registryId)
await registry.decode(payload, {
  [SchemaType.AVRO]: {
    readerSchema
  }
})

This also ensures that the same schema type options from the constructor are used when constructing the Type to use for decoding (i.e. the avsc options).

FYI @ivan-klass

@Nevon Nevon merged commit 8ebd46b into master Nov 22, 2021
@Nevon Nevon deleted the avro-schema-resolution branch November 22, 2021 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants