Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CTE materialization internal error: Fails with 0 length varchar #21791

Open
jaystarshot opened this issue Jan 25, 2024 · 0 comments
Open

CTE materialization internal error: Fails with 0 length varchar #21791

jaystarshot opened this issue Jan 25, 2024 · 0 comments

Comments

@jaystarshot
Copy link
Member

jaystarshot commented Jan 25, 2024

This is because Presto allows 0 length varchar but hive doesn't. When we materialize this CTE it fails but without materialization, it succeeds.

@Test
   public void testCteWithZeroLengthVarchar()
   {
       String testQuery = "WITH temp AS (" +
               "  SELECT * FROM (VALUES " +
               "    (CAST('' AS VARCHAR(0)), 9)" +
               "  ) AS t (text_column, number_column)" +
               ") SELECT * FROM temp";
       QueryRunner queryRunner = getQueryRunner();
       compareResults(queryRunner.execute(getMaterializedSession(),
                       testQuery),
               queryRunner.execute(getSession(),
                       testQuery));
   }

Stack Trace

Caused by: java.lang.RuntimeException: Varchar length 0 out of allowed range [1, 65535]
at org.apache.hadoop.hive.serde2.typeinfo.BaseCharUtils.validateVarcharParameter(BaseCharUtils.java:32)
at org.apache.hadoop.hive.serde2.typeinfo.VarcharTypeInfo.(VarcharTypeInfo.java:33)
at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.createPrimitiveTypeInfo(TypeInfoFactory.java:159)
at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getPrimitiveTypeInfo(TypeInfoFactory.java:117)
at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getVarcharTypeInfo(TypeInfoFactory.java:183)
at com.facebook.presto.hive.HiveTypeTranslator.translate(HiveTypeTranslator.java:98)
at com.facebook.presto.hive.HiveType.toHiveType(HiveType.java:218)
at com.facebook.presto.hive.HiveMetadata.getColumnHandles(HiveMetadata.java:3584)
at com.facebook.presto.hive.HiveMetadata.createTemporaryTable(HiveMetadata.java:1107)

So we might need this check for the 0 length Varchar due to , because write with 0 length varchar fails.

See reference - While Presto supports Varchar of length 0 (as discussed in trinodb/trino#1136

Your Environment

  • Presto version used:
  • Storage (HDFS/S3/GCS..):
  • Data source and connector used:
  • Deployment (Cloud or On-prem):
  • Pastebin link to the complete debug logs:

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce

Screenshots (if appropriate)

Context

@jaystarshot jaystarshot changed the title CTE materialization internal error: CTE Materialization fails with 0 length varchar CTAS Write and CTE materialization internal error: Fails with 0 length varchar Jan 25, 2024
@jaystarshot jaystarshot changed the title CTAS Write and CTE materialization internal error: Fails with 0 length varchar CTE materialization internal error: Fails with 0 length varchar Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🆕 Unprioritized
Development

No branches or pull requests

1 participant