[Fuzzer] Missing Unicode character range at UTF8 validation

[PedroTadim]

What happens?

This morning, the fuzzer triggered another assertion error:

SELECT upper(decode('\xF6\x96\xB0\x85'::BLOB)::VARCHAR);

src/function/scalar/string/caseconvert.cpp:64: duckdb::idx_t duckdb::GetResultLength(const char *, duckdb::idx_t) [IS_UPPER = true]: Assertion `new_sz >= 0' failed.

After some digging, I see Utf8Proc::Analyze is missing validation for Unicode characters not representable by UTF8 as described here: https://en.wikipedia.org/wiki/UTF-8#cite_note-16

This is an issue from my side as well because I shouldn't generate these strings either :) I am going to add the missing validation.

To Reproduce

Run the query above.

OS:

Linux

DuckDB Version:

latest from sources

DuckDB Client:

Shell

Full Name:

Pedro Ferreira

Affiliation:

Huawei

Have you tried this on the latest master branch?

• I agree

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

• I agree

[Mytherin]

Fixed by #4711