Support reading Avro files in zstd codec

[siumingdev]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I would like to read Avro files in ztsd codec, like this

from datafusion import SessionContext

ctx = SessionContext()
ctx.read_avro("/path/to/my/avro/in/zstd/codec")

But currently it gives the following error:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[4], line 4
      1 from datafusion import SessionContext
      3 ctx = SessionContext()
----> 4 ctx.read_avro("/path/to/my/avro/in/zstd/codec")

Exception: DataFusion error: AvroError(CodecNotSupported("zstandard"))

I am running the code using official Python 3.9 docker image (python:3.9-slim) and install using pip install datafusion.

Describe the solution you'd like
No idea, is it even not supported in the original Rust implementation?

Describe alternatives you've considered
Read the file into using other Avro libraries and convert into datafusion dataframes.

Additional context

[mesejo]

As you mentioned, this is not supported by the Rust Implementation (see the reading options). Is better to move the ticket to the appropriate repo. What are your thoughts @alamb @andygrove?

[alamb]

I agree -- this is a problem lower down in the stack (either in DataFusion or avro_rs)

I did a brief look at avro_rs and didn't see any mention of zstd 🤔 https://docs.rs/avro-rs/latest/avro_rs/#using-codecs-to-compress-data

[timsaucer]

I have tested this against DF42.0. I created a avro file using fastavro and zstandard to compress it. I read it and displayed the data successfully. Please reopen an issue if you continue to have a problem.

Thank you @drauschenbach for pointing out this issue.