Skip to content

Handle binary varying type for newer redshift driver #25488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

adkharat
Copy link
Contributor

@adkharat adkharat commented Jul 3, 2025

Description

Redshift driver version is 2.1.0.32

JdbcTypeHandle {
   jdbcType=1111, 
   jdbcTypeName=binary varying, 
   columnSize=2147483647, 
   decimalDigits=0
}

Redshift driver version is 2.1.0.28

JdbcTypeHandle {
    jdbcType = -4,
    jdbcTypeName = "varbyte",
    columnSize = 0,
    decimalDigits = 0,
    arrayDimensions = null
}

Newer redshift drivers uses Types.OTHER (1111) to represent VARBYTE and describes it as "binary varying"

JDBC Driver Version Reported JDBC Type JDBC Type Name
2.1.0.28 -4 (LONGVARBINARY) "varbyte"
2.1.0.32 1111 (OTHER) "binary varying"

Motivation and Context

create table redshift.test.redtest4 (varbincolumn varbinary);

The Redshift JDBC driver returns the JDBC type for VARBYTE columns as:
JdbcTypeHandle{jdbcType=1111, jdbcTypeName=binary varying, columnSize=2147483647, decimalDigits=0}

This type is not recognized by Presto's default jdbcTypeToPrestoType() mapping,
resulting in the column being ignored as unsupported during DESCRIBE or INSERT queries

DESCRIBE redshift.test.redtest1; output is:

presto> DESCRIBE redshift.test.redtest1;
 Column | Type | Extra | Comment 
--------+------+-------+---------
(0 rows)

Query 20250626_070357_00001_29zdf, FINISHED, 1 node
Splits: 19 total, 19 done (100.00%)
[Latency: client-side: 0:36, server-side: 0:36] [0 rows, 0B] [0 rows/s, 0B/s]

SHOW CREATE TABLE redshift.test.redtest1; output is:

presto> SHOW CREATE TABLE redshift.test.redtest1;
Query 20250626_070441_00002_29zdf failed: Table 'test.redtest1' has no supported columns (all 1 columns are not supported)

INSERT query output is:

presto> insert into redshift.test.redtest1 (varbincolumn) values (cast('Test varbinary' as varbinary));
Query 20250626_071129_00006_29zdf failed: Table 'test.redtest1' has no supported columns (all 1 columns are not supported)

Impact

Test Plan

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

Redshift Connector Changes
* Fix Redshift VARBYTE column handling for JDBC driver version 2.1.0.32+ by mapping jdbcType=1111 and jdbcTypeName="binary varying" to Presto's VARBINARY type.

@adkharat adkharat requested a review from a team as a code owner July 3, 2025 10:59
@prestodb-ci prestodb-ci added the from:IBM PR from IBM label Jul 3, 2025
@prestodb-ci prestodb-ci requested review from a team, infvg and imsayari404 and removed request for a team July 3, 2025 10:59
@adkharat adkharat changed the title handle binary varying type for newer redshift driver Handle binary varying type for newer redshift driver Jul 3, 2025
Copy link
Member

@agrawalreetika agrawalreetika left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM.
Please update the commit message with -

Handle binary varying type introduced in newer Redshift drivers

@adkharat adkharat force-pushed the handle_binary_varying_in_redshift branch from bddbdc8 to bdad685 Compare July 3, 2025 16:16
agrawalreetika
agrawalreetika previously approved these changes Jul 3, 2025
@adkharat
Copy link
Contributor Author

adkharat commented Jul 7, 2025

@tdcmeehan can you please review and the merge this PR?

{
String typeName = typeHandle.getJdbcTypeName();

if (typeName.equalsIgnoreCase("binary varying")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a comment explaining that this is dependent on the version of the driver, as is included in this PR?

Copy link
Contributor Author

@adkharat adkharat Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tdcmeehan 1 line comment added.

or do you want me to add comment in Javadoc-style format

/**
 * Maps Redshift VARBYTE columns to Presto VARBINARY.
 * <p>
 * Redshift JDBC driver version ≥ 2.1.0.32 reports VARBYTE as JDBC type 1111 (Types.OTHER)
 * with type name "binary varying", whereas older versions (e.g., 2.1.0.28) report it
 * as type -4 (Types.LONGVARBINARY) with type name "varbyte".
 * This mapping ensures compatibility with newer driver versions.
 */

@adkharat adkharat force-pushed the handle_binary_varying_in_redshift branch from f314b4f to 7e0ae2d Compare July 8, 2025 06:20
@adkharat adkharat requested a review from tdcmeehan July 8, 2025 06:22
@adkharat adkharat force-pushed the handle_binary_varying_in_redshift branch from 7e0ae2d to e036ddc Compare July 8, 2025 14:37
@agrawalreetika agrawalreetika merged commit 252d95f into prestodb:master Jul 8, 2025
108 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
from:IBM PR from IBM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants