Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-11691: Support VARBINARY and LONGVARBINARY types in PutDatabaseRecord #7380

Closed
wants to merge 2 commits into from

Conversation

mattyb149
Copy link
Contributor

Summary

NIFI-11691 This PR adds support for the java.sql.Types VARBINARY and LONGVARBINARY in PutDatabaseRecord. It converts the record field value to byte[] and then calls setBytes() when setting the parameter value in the PreparedStatement

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 11
    • JDK 17

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing this issue @mattyb149, the general approach looks good, but I noted a couple questions about the alternative type handling.

try {
ps.setBytes(index, byteArray);
} catch (SQLException e) {
throw new IOException("Unable to parse binary data " + value, e.getCause());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the potential size of these objects, recommend replacing value the array length to avoid extremely large log messages.

try {
ps.setBytes(index, value.toString().getBytes(StandardCharsets.UTF_8));
} catch (SQLException e) {
throw new IOException("Unable to parse binary data " + value, e.getCause());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted above, recommend replacing value with the length of the bytes.

}
} else {
try {
ps.setBytes(index, value.toString().getBytes(StandardCharsets.UTF_8));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth adding a comment on this line, since value.toString() may not provide an expected representation of the underlying object type. Perhaps there should be some additional type checking in this conditional to determine if value is an array type or collection type, otherwise it seems like this could produce unexpected results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just in case the data came in as a String

Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the adjustments @mattyb149! +1 merging

exceptionfactory pushed a commit that referenced this pull request Jun 15, 2023
This closes #7380

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 7609499)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants