Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert homogeneously typed Literals into ENUMs #596

Closed
kevinjacobs-delfi opened this issue Apr 10, 2024 · 3 comments
Closed

Convert homogeneously typed Literals into ENUMs #596

kevinjacobs-delfi opened this issue Apr 10, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@kevinjacobs-delfi
Copy link
Contributor

Is your feature request related to a problem? Please describe.
I am translating a large corpus of Avro schemas into Pydantic V2 models and our convention is to encode simple string enums as Literal types in those models. Currently, the dataclasses-avroschema translates homogeneously typed Literals into a field of that type.

e.g.

class A(AvroBaseModel):
    foo: Literal["a", "b", "c"]

yields this schema:

{
    "type": "record", 
    "name": "A", 
    "fields": [{"name": "foo", "type": "string"}]
}

Describe the solution you'd like
The current behavior is backward compatible, since it widens the range of valid values. But it also means that the constraint on values is lost for other users of these schemas unless using them mediated via our Pydantic models. It would be great if we could add an config value or annotation that changes this behavior locally or globally to generate Avro ENUMs for such Literals.

{
    "type": "record", 
    "name": "A", 
    "fields": [{"name": "foo", "type": {"type": "enum", "name": "FooValues", "symbols": ["a", "b", "c"]}}]
}

Describe alternatives you've considered
We could, in theory, switch from using Literal to using StrEnum, but would require updating a huge amount of related code to do so. Our data governance team would probably recommend switching to another Avro/Pydantic mapping library instead.

@marcosschroh
Copy link
Owner

Hi @kevinjacobs-delfi

It makes sense that Literals with ONLY strings primitive values are represented as enum in avro. This might require some extra work to make sure that conversion from avro schemas to models also works, so avro enums that were generated with pythonn Literals will generate Literals rather then python Enum.

@marcosschroh
Copy link
Owner

II have added a new option called convert_literal_to_enum to convert Literal strings to avro enum.

@kevinjacobs-delfi
Copy link
Contributor Author

Great! I'll give it a try and let you know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants