Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to parse two-dimensional nested array of ints #35

Open
kevinashaw opened this issue Dec 2, 2022 · 1 comment
Open

Failing to parse two-dimensional nested array of ints #35

kevinashaw opened this issue Dec 2, 2022 · 1 comment

Comments

@kevinashaw
Copy link

kevinashaw commented Dec 2, 2022

Just found this project today and it has been very helpful, but I ran into a problem with nested arrays.
The following avro schema:

{
    "namespace": "com.company.code",
    "type": "record",
    "name": "MyAvroSchema",
    "doc" : "...",
    "fields": [
        {
            "name": "MyArray",
            "type": {
                "type": "array",
                "items": {
                    "type": {"type": "array","items": "int"}
                }
            }
        }
    ]
}

generates the following error:

Traceback (most recent call last):
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/Users/user/Library/Application Support/JetBrains/Toolbox/apps/PyCharm-P/ch-0/222.4459.20/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/Users/user/Library/Application Support/JetBrains/Toolbox/apps/PyCharm-P/ch-0/222.4459.20/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/user/arrhythmia/file_schemas/validate_schema.py", line 5, in <module>
    parsed_schema = schema.parse()
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/schema.py", line 28, in parse
    return RecordType.build(schema, skip_extra_keys=skip_extra_keys)
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 812, in build
    record_type.__fields = {
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 813, in <dictcomp>
    field['name']: RecordTypeField.build(
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 585, in build
    field.__type = cls.__build_field_type(json_repr, custom_fields)
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 545, in __build_field_type
    return cls._get_field_from_json(json_repr['type'], custom_fields)
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 210, in _get_field_from_json
    return getattr(sys.modules[__name__], FIELD_MAPPING[field_type['type']]).build(field_type, custom_fields)
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 1016, in build
    array_type.__items = ArrayType._get_field_from_json(json_repr['items'], custom_fields)
  File "/Users/user/miniconda3/envs/arrhythmia/lib/python3.9/site-packages/avro_validator/avro_types.py", line 210, in _get_field_from_json
    return getattr(sys.modules[__name__], FIELD_MAPPING[field_type['type']]).build(field_type, custom_fields)
TypeError: unhashable type: 'dict'

If I remove the inner array it parses correctly.

@kevinashaw
Copy link
Author

As discussed here in StackOverflow, there was an error in my schema. That said, the avro_validator code may wish to handle this particular case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant