Skip to content

[Bug] Reading a JSON column with no string cast fails with a message-less NullPointerException #9587

Description

@LuciferYang

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

master, 2788fe596 (2.1-SNAPSHOT).

Compute Engine

Flink and Spark, reading a format table with 'file.format' = 'json' that has a column whose type has no cast from string.

Minimal reproduce step

Read a JSON format table with a MULTISET<STRING> column:

CREATE TABLE t (f0 MULTISET<STRING>) WITH (
    'type' = 'format-table',
    'file.format' = 'json',
    'path' = '...'
);
SELECT * FROM t;

The query fails with a NullPointerException that has no message. JsonFileReader.convertPrimitiveStringToType:

default:
    BinaryString binaryString = BinaryString.fromString(str);
    CastExecutor cast = CastExecutors.resolve(DataTypes.STRING(), dataType);
    return cast.cast(binaryString);

CastExecutors.resolve is documented as returning null when no rule can be resolved, and there is no rule from STRING to MULTISET, VARIANT or BLOB. The result is dereferenced on the next line.

What doesn't meet your expectations?

The format already has a sentence for this case: JsonFileFormat.validateDataType throws UnsupportedOperationException("Unsupported data type for JSON format: X") at create time. Reading the same type should say the same thing rather than producing a stack trace with no message, which sends the reader looking at the data rather than at the column type.

A managed table never gets here, since SchemaValidation rejects those types when the table is created. A format table is created through CatalogUtils.validateCreateTable, which does not call validateDataFields, so the column exists and the failure surfaces on read.

Anything else?

The same shape, dereferencing CastExecutors.resolve without a null check, also exists in ArrayToStringCastRule, MapToStringCastRule, RowToStringCastRule and InternalRowPartitionComputer. I have not checked whether those are reachable, so I am not proposing to change them here.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions