'Malformed string' or 'Cannot transliterate character between character sets' with non ASCII char

[Fab8573]

Hello,`

Issue: "Malformed String" with Non-ASCII Characters in Numeric or Timestamp Fields on Firebird Database

This issue occurs in any version of Firebird when using non-ASCII characters in a numeric or timestamp field.

It can be easily reproduced using the sample EMPLOYEE.FDB database :

SELECT * FROM EMPLOYEE r where hire_date like '%€%'

• If the database charset is NONE: The query results in a 'Malformed String' error.
• If the database charset is UTF8: The query results in an 'Arithmetic exception, numeric overflow, or string truncation: Cannot transliterate character between character sets' error.

The same problem occurs with any non-ASCII character for example '£', 'é' or '²' etc...

However, using a standard ASCII string works fine:

SELECT * FROM EMPLOYEE r where hire_date like '%ytrytrytyn$trbtrytr$$$ybrtjury%'
=> Work fine

[aafemt]

Database charset in this case is irrelevant. Strings are converted using connection charset. What was connection charset in your examples?

[Fab8573]

With default EMPLOYEE.FDB
if Connection Charset = NONE error is 'Malformed String'
if Connection Charset = UTF8 or Charset = WIN1252 error is 'Arithmetic exception, numeric overflow, or string truncation: Cannot transliterate character between character sets'

Same problem if EMPLOYEE.FDB is recreated with UTF8 or WIN1252 character sets. Error will be 'Arithmetic exception, numeric overflow, or string truncation: Cannot transliterate character between character sets'

Problem is : In any case Firebird try do an ASCII conversion for non alphanumeric fields comparison. Why ?