Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-2531: Fixed JdbcCommon handling of BigInteger objects for Avro #823

Closed
wants to merge 1 commit into from

Conversation

mattyb149
Copy link
Contributor

No description provided.

@bbende
Copy link
Contributor

bbende commented Aug 10, 2016

Reviewing...

@bbende
Copy link
Contributor

bbende commented Aug 10, 2016

Tested this out with MySQL, created a table like the following:

mysql> CREATE TABLE BIGINT_TEST (id bigint(20) unsigned, name varchar(255));
Query OK, 0 rows affected (0.02 sec)

mysql> insert into BIGINT_TEST (id, name) values (22222222222222222222, "test");
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> select * from BIGINT_TEST;
+----------------------+------+
| id                   | name |
+----------------------+------+
| 18446744073709551615 | test |
+----------------------+------+
1 row in set (0.00 sec)

Verified I got the error before your patch, then applied it and got a different error:

java.lang.ArithmeticException: BigInteger out of long range
    at java.math.BigInteger.longValueExact(BigInteger.java:4383) ~[na:1.8.0_74]
    at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:125) ~[na:na]

I understand there isn't much we can do here because the value is too big for a long, but would we better off representing BIGINT as a string in the schema and output so that we never run into an error?

I realize we lose the typing then, but not sure if having some of the data typed and some with errors is better or worse.

@mattyb149
Copy link
Contributor Author

Good point, let me see if I can use the column width (for BIGINT types) to determine whether to expect a long or string in the value.

@bbende
Copy link
Contributor

bbende commented Aug 10, 2016

Latest update looks good, nice work! Will merge to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants