Bug description
String.toLowerCase() and String.toUpperCase() follow the JVM default locale. Under a Turkish or Azeri default, i uppercases to the dotted İ and I lowercases to the dotless ı, so any token that is case-folded before being matched, parsed or signed stops matching what it is compared against.
This started as an identifier-matching bug. With case-sensitive = false, CDC table mapping, computed columns and the Arrow readers all route through StringUtils.toLowerCaseIfNeed, and CdcRecord.fieldNameLowerCase is the record side of the same join, so a source column INDEX stops mapping to schema field index and the column silently nulls out.
Auditing the rest of paimon-api, paimon-common and paimon-core turned up the same conversion in places where it throws rather than mismatching quietly:
CoreOptions.partitionMarkDoneActions() does PartitionMarkDoneAction.valueOf(x.replace('-','_').toUpperCase()). Both success-file and done-partition contain an i, so under Turkish this throws IllegalArgumentException: No enum constant ...PartitionMarkDoneAction.SUCCESS_FİLE.RowKind.fromShortString("+i") uppercases to +İ, matches no case arm, and throws UnsupportedOperationException.DistributedLockDialectFactory does JdbcProtocol.valueOf(protocol.toUpperCase()) and loses SQLITE and MARIADB, so JDBC catalog locking cannot resolve its dialect.OperatingSystem lowercases os.name and then looks for solaris, which becomes solarıs, so the OS is reported as UNKNOWN.- The DLF request signers lowercase header names to build the signature, option key lookups and format identifiers lowercase user-supplied keys, and system table lookup lowercases the table name. All of these compare the result against a fixed ASCII string.
Expected behavior
None of these conversions should depend on the operator's locale: they are all machine-facing, so they should pin Locale.ROOT.
BinaryString.toLowerCase / toUpperCase are already safe: the ASCII path uses Character.toLowerCase and the non-ASCII fallback pins Locale.ROOT, so the SQL upper() / lower() transforms over user data are not affected.
Bug description
String.toLowerCase()andString.toUpperCase()follow the JVM default locale. Under a Turkish or Azeri default,iuppercases to the dottedİandIlowercases to the dotlessı, so any token that is case-folded before being matched, parsed or signed stops matching what it is compared against.This started as an identifier-matching bug. With
case-sensitive = false, CDC table mapping, computed columns and the Arrow readers all route throughStringUtils.toLowerCaseIfNeed, andCdcRecord.fieldNameLowerCaseis the record side of the same join, so a source columnINDEXstops mapping to schema fieldindexand the column silently nulls out.Auditing the rest of
paimon-api,paimon-commonandpaimon-coreturned up the same conversion in places where it throws rather than mismatching quietly:CoreOptions.partitionMarkDoneActions()doesPartitionMarkDoneAction.valueOf(x.replace('-','_').toUpperCase()). Bothsuccess-fileanddone-partitioncontain ani, so under Turkish this throwsIllegalArgumentException: No enum constant ...PartitionMarkDoneAction.SUCCESS_FİLE.RowKind.fromShortString("+i")uppercases to+İ, matches no case arm, and throwsUnsupportedOperationException.DistributedLockDialectFactorydoesJdbcProtocol.valueOf(protocol.toUpperCase())and losesSQLITEandMARIADB, so JDBC catalog locking cannot resolve its dialect.OperatingSystemlowercasesos.nameand then looks forsolaris, which becomessolarıs, so the OS is reported asUNKNOWN.Expected behavior
None of these conversions should depend on the operator's locale: they are all machine-facing, so they should pin
Locale.ROOT.BinaryString.toLowerCase/toUpperCaseare already safe: the ASCII path usesCharacter.toLowerCaseand the non-ASCII fallback pinsLocale.ROOT, so the SQLupper()/lower()transforms over user data are not affected.