[FLINK-39053][format/avro] Support VARCHAR to ENUM conversion in avro-confluent format

[ddebowczyk92]

What is the purpose of the change

This PR fixes an AvroTypeException that occurs when writing Flink VARCHAR fields to Kafka using the avro-confluent format where the target Avro schema defines those fields as ENUMs.

Brief change log

• Modified RowDataToAvroConverter VARCHAR case to check target schema type
• Added enum symbol conversion when target Avro schema is ENUM type
• Maintains backward compatibility by defaulting to Utf8 for non-enum schemas
• Enables round-trip processing of Avro data with ENUM fields through Flink SQL

Verifying this change

This change can be verified manually by:

1. Creating a Kafka source with avro-confluent format containing ENUM field
2. Reading the data (Flink will represent ENUM as VARCHAR)
3. Creating a Kafka sink with the same Avro schema (containing ENUM)
4. Writing the data back to Kafka
5. Verifying successful write without AvroTypeException

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: yes
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: (no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[flinkbot]

CI report:

• 865fa6b Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[rionmonster]

@ddebowczyk92

Thanks for the contribution!

We’ll likely want to introduce a unit test to verify this enum behavior, similar to some of the other conversion-based tests in AvroRowDataDeSerializationSchemaTest.java. There doesn’t appear to be any existing previous coverage there for enums, so adding one would make it easy to verify this behavior and help guard against future enum-related regressions.

[davidradl]

could we have a unit test for this at the fix time please?

[davidradl]

The title talks about avro-confuent format but the code appears to be in the arvo section so effects the avro format at well.