Improve Avro implementation #332
Description
The current Avro writer and reader implementations are not that efficient because we need to use avro::GenericDatum as the intermediate layer:
- Avro writer:
ArrowArray->avro::GenericDatum->avro::Encoder - Avro reader:
avro::Decoder->avro::GenericDatum->ArrowArray
It would be more efficient if we can get rid of conversion from/to avro::GenericDatum by directly using the encoder/decoder in the Avro writer/reader. It can greatly improve the performance of manifest file I/O.