NIFI-7159 - Add support for some data types in various processors#4104
NIFI-7159 - Add support for some data types in various processors#4104Xeinn wants to merge 7 commits intoapache:masterfrom
Conversation
|
Will start reviewing tonight. |
|
You have a checkstyle violation in your changes |
|
Getting build errors now related to Decimal128. |
|
I think we can use this and a little reflection to get around that one explicit reference to Decimal128 https://stackoverflow.com/questions/52870627/get-mongodb-driver-version-programmatically |
|
Yeah, the javadocs are sketchy on that class, but there's gotta be some way where we can look at the driver, find it out if it's |
|
Hmmm not finding anything that I would class as a reliable source for driver version that’s available from the Mongo Driver.
I may be overlooking something obvious however.
There is the option of a slightly more implied check by verifying the presence of Decimal128?
Will pick up in the morning when I’ve got a bit more coffee in me 😊 thanks for the help so far, I’m sure we’ll have it cracked by tomorrow.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
…________________________________
From: Mike <notifications@github.com>
Sent: Tuesday, March 3, 2020 12:54:54 AM
To: apache/nifi <nifi@noreply.github.com>
Cc: Chris Manniex <cmanniex@hotmail.com>; Author <author@noreply.github.com>
Subject: Re: [apache/nifi] NIFI-7159 (#4104)
Yeah, the javadocs are sketchy on that class, but there's gotta be some way where we can look at the driver, find it out if it's >= 3.8 and then use reflection so that older driver support will still work.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%3Femail_source%3Dnotifications%26email_token%3DAFHXE7DNUOFVMERHDXAS6RDRFRIN5A5CNFSM4K72OIGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENRUWWY%23issuecomment-593709915&data=02%7C01%7C%7Cb00f111ed1784593059008d7bf0d7dfe%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637187936968703202&sdata=gZolhXppkoQcrBUpKALLpcQ%2BKe8ASo7QAjoL16BsmyU%3D&reserved=0>, or unsubscribe<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFHXE7GHAMKDIBXIW7W3YQTRFRIN5ANCNFSM4K72OIGA&data=02%7C01%7C%7Cb00f111ed1784593059008d7bf0d7dfe%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637187936968713213&sdata=9nnXRNKpEr7Gd7j61o%2Fi7w2%2Bs0oe1708PJvXHPrEqpk%3D&reserved=0>.
|
MikeThomsen
left a comment
MikeThomsen
left a comment
There was a problem hiding this comment.
Looks pretty good, but there is definitely some scope creep that will require others' input.
| } | ||
| break; | ||
| } | ||
| case DECIMAL: |
There was a problem hiding this comment.
I think this should actually be a double because that's the maximum that Elasticsearch supports. See here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html
| retVal = clientService.toBytes(record.getAsBoolean(field)); | ||
| break; | ||
| case DECIMAL: | ||
| // Decimal to be treated as the same as double |
There was a problem hiding this comment.
It should be broken down into a byte array like the other types.
| return new DoubleWritable((double) o); | ||
| } | ||
| // Map BigDecimal to a Double type - this should be improved to map to Hive Decimal type | ||
| if (o instanceof BigDecimal) { |
There was a problem hiding this comment.
There were some runtime issues with unit tests related to Orc. @mattyb149 @bbende @ijokarumawak do y'all have any insight into this ORC conversion?
| case STRING: | ||
| case DECIMAL: | ||
| fields.add(new RecordField(cs.getName(), RecordFieldType.STRING.getDataType())); | ||
| fields.add(new RecordField(cs.getName(), RecordFieldType.DECIMAL.getDataType())); |
There was a problem hiding this comment.
I don't know anything about Kudu, so I may put this ticket on hold and ask for input from nifi-dev on a few of these other components (ex Hive)
|
|
||
| @Override | ||
| public String toString() { | ||
| return "DECIMAL" + Integer.toString(precision) + Integer.toString(scale); |
There was a problem hiding this comment.
Might be a good idea to add underscores (for example) so it looks like this:
DECIMAL_11_11
instead of this:
DECIMAL1111
Btw Integer.toString is redundant with the string literal plus '+' operator (makes it a bit less readable).
| } | ||
|
|
||
| public static BigDecimal toBigDecimal(final Object value, final String fieldName) { | ||
| if (value == null) { |
There was a problem hiding this comment.
Shouldn't we cover all the Number cases?
I think new BigDecimal(value.toString()); might be enough.
(Can't think of an example where the string representation of a Number would not be good.)
| /** | ||
| * Returns a data type that represents a DECIMAL value with given precision and scale | ||
| * | ||
| * @param precision indicates the expected number of digits before the decimal point |
There was a problem hiding this comment.
Precision is the number of all digits.
| import java.io.InputStream; | ||
| import java.io.Reader; | ||
| import java.lang.reflect.Array; | ||
| import java.math.BigDecimal; |
There was a problem hiding this comment.
Currently
DataTypeUtils.findMostSuitableType(
new BigDecimal("11.110"),
Arrays.asList(RecordFieldType.DECIMAL.getDecimalDataType(5, 3)),
Function.identity()
);correctly returns Optional.of(RecordFieldType.DECIMAL.getDecimalDataType(5, 3));, but
DataTypeUtils.findMostSuitableType(
new BigDecimal("1.110"),
Arrays.asList(RecordFieldType.DECIMAL.getDecimalDataType(5, 3)),
Function.identity()
);("1.110" instead of "11.111") returns Optional.empty();
This is because the inferred type DECIMAL(4,3) is not equal to the provided DECIMAL(5,3) so the getWiderType method is called which doesn't handle cases where the two types are the same in general, only differ in their details.
Probably the public static Optional<DataType> getWiderType(final DataType thisDataType, final DataType otherDataType) should handle two different DECIMAL types by returning another DECIMAL that is the superset of the two in terms of scale and precision.
|
Thanks for the feedback.
Good idea will be making some changes as per Mike’s feedback later in the week so will incorporate the below then.
Regards
Chris
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
________________________________
From: tpalfy <notifications@github.com>
Sent: Friday, March 6, 2020 7:31:43 PM
To: apache/nifi <nifi@noreply.github.com>
Cc: Chris Manniex <cmanniex@hotmail.com>; Author <author@noreply.github.com>
Subject: Re: [apache/nifi] NIFI-7159 (#4104)
@tpalfy requested changes on this pull request.
________________________________
In nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/type/DecimalDataType.java<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r389051423&data=02%7C01%7C%7Cb6ae8c600b2441190d0408d7c20500ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637191199047595864&sdata=yejHbBKmWv0PfQHtTJFaDrgRy0NCyByfe%2FCSbLI1cJU%3D&reserved=0>:
+ return true;
+ }
+ if (obj == null) {
+ return false;
+ }
+ if (!(obj instanceof DecimalDataType)) {
+ return false;
+ }
+
+ final DecimalDataType other = (DecimalDataType) obj;
+ return precision == other.getPrecision() && scale == other.getScale();
+ }
+
+ @OverRide
+ public String toString() {
+ return "DECIMAL" + Integer.toString(precision) + Integer.toString(scale);
Might be a good idea to add underscores (for example) so it looks like this:
DECIMAL_11_11
instead of this:
DECIMAL1111
Btw Integer.toString is redundant with the string literal plus '+' operator (makes it a bit less readable).
________________________________
In nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r389052612&data=02%7C01%7C%7Cb6ae8c600b2441190d0408d7c20500ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637191199047605858&sdata=oAr4Ue8tgmCg0LLB5mi1yghxl9kYgY1OblG9YLjyfTs%3D&reserved=0>:
@@ -1237,10 +1246,38 @@ public static Double toDouble(final Object value, final String fieldName) {
throw new IllegalTypeConversionException("Cannot convert value [" + value + "] of type " + value.getClass() + " to Double for field " + fieldName);
}
+ public static BigDecimal toBigDecimal(final Object value, final String fieldName) {
+ if (value == null) {
Shouldn't we cover all the Number cases?
I think new BigDecimal(value.toString()); might be enough.
(Can't think of an example where the string representation of a Number would not be good.)
________________________________
In nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/RecordFieldType.java<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r389092840&data=02%7C01%7C%7Cb6ae8c600b2441190d0408d7c20500ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637191199047605858&sdata=oOrZRC7UnSHPpsuDAe51e6gy2O%2BgHY%2BIe%2Bk1hff0piM%3D&reserved=0>:
@@ -280,6 +286,25 @@ public DataType getRecordDataType(final RecordSchema childSchema) {
return new RecordDataType(childSchema);
}
+ /**
+ * Returns a data type that represents a DECIMAL value with given precision and scale
+ *
+ * @param precision indicates the expected number of digits before the decimal point
Precision is the number of all digits.
________________________________
In nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23discussion_r389097315&data=02%7C01%7C%7Cb6ae8c600b2441190d0408d7c20500ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637191199047615853&sdata=ztQpc0iqHV7rQndHeBKvWxQhosmnOOxINZfGdv910os%3D&reserved=0>:
@@ -34,6 +34,7 @@
import java.io.InputStream;
import java.io.Reader;
import java.lang.reflect.Array;
+import java.math.BigDecimal;
Currently
DataTypeUtils.findMostSuitableType(
new BigDecimal("11.110"),
Arrays.asList(RecordFieldType.DECIMAL.getDecimalDataType(5, 3)),
Function.identity()
);
correctly returns Optional.of(RecordFieldType.DECIMAL.getDecimalDataType(5, 3));, but
DataTypeUtils.findMostSuitableType(
new BigDecimal("1.110"),
Arrays.asList(RecordFieldType.DECIMAL.getDecimalDataType(5, 3)),
Function.identity()
);
("1.110" instead of "11.111") returns Optional.empty();
This is because the inferred type DECIMAL(4,2) is not equal to the provided DECIMAL(5,2) so the getWiderType method is called which doesn't handle cases where the two types are the same in general, only differ in their details.
Probably the public static Optional<DataType> getWiderType(final DataType thisDataType, final DataType otherDataType) should handle two different DECIMAL types by returning another DECIMAL that is the superset of the two in terms of scale and precision.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%3Femail_source%3Dnotifications%26email_token%3DAFHXE7DKUBOFT4EWJBAGEADRGFFR7A5CNFSM4K72OIGKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCYKZB2Y%23pullrequestreview-370512107&data=02%7C01%7C%7Cb6ae8c600b2441190d0408d7c20500ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637191199047615853&sdata=LU88VM5%2Fw%2FMrEzyJUdVu5yOxlkVlwOg2sRdz5JmyvGo%3D&reserved=0>, or unsubscribe<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFHXE7EJCF5ODMWQD2DE64LRGFFR7ANCNFSM4K72OIGA&data=02%7C01%7C%7Cb6ae8c600b2441190d0408d7c20500ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637191199047615853&sdata=f2DOBtkZihGP3GAS%2FKlmBymYTT%2B6lVgI7RWjo74SipY%3D&reserved=0>.
|
pvillard31
changed the title
|
@Xeinn there's a merge conflict now. Instructions follow if you're newish to git. Otherwise ignore. To resolve do the following:
Don't git pull apache/nifi master into your branch. |
|
Thanks Mike,
That’ll help.
Will tackle it this evening and shout if I have problems/questions
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
…________________________________
From: Mike <notifications@github.com>
Sent: Sunday, March 15, 2020 1:21:09 PM
To: apache/nifi <nifi@noreply.github.com>
Cc: Chris Manniex <cmanniex@hotmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [apache/nifi] NIFI-7159 - Add support for some data types in various processors (#4104)
@Xeinn<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FXeinn&data=02%7C01%7C%7C2263074bc9c546305bd208d7c8e3ba1a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637198752708391176&sdata=QP6Ul1JuqCThkQQg3S01T8aZqsjpe1roKp0QiBetcqw%3D&reserved=0> there's a merge conflict now. Instructions follow if you're newish to git. Otherwise ignore.
To resolve do the following:
1. Pull apache/nifi master into your master.
2. Rebase (git rebase master) your branch off of master again.
3. Follow the prompts that git gives you. Be careful to read what it says.
4. git push origin --force BRANCH.
Don't git pull apache/nifi master into your branch.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23issuecomment-599209694&data=02%7C01%7C%7C2263074bc9c546305bd208d7c8e3ba1a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637198752708401167&sdata=qtGlAFv8ejfLeZ86havjZ0d7n3A9ss4pXT0mNNPdhSs%3D&reserved=0>, or unsubscribe<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFHXE7CDMKCJ5NRHQYJ2C73RHTI4LANCNFSM4K72OIGA&data=02%7C01%7C%7C2263074bc9c546305bd208d7c8e3ba1a%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637198752708411162&sdata=hYnvlM5lYzNdYTUaM%2BpZjC2BzBC1nQEEoCTEq1ZP6eA%3D&reserved=0>.
|
Aligning the implementation of DECIMAL RecordFieldType with other types Introduce RecordFieldType of DECIMAL for BigDecimal values Adding RecordFieldType DECIMAL to ResultSetRecordSet Introduce RecordFieldType DECIMAL into DataTypeUtils Added missing BIGINT type to convertSimpleIfPossible method Added missing type BIGINT to convertField method Added missing type BIGINT to writeFieldForType Added missing type BIGINT to parseFieldForType Added missing type BIGINT to parseStringForType Potentially breaking change of behaviour for Kudu Service. Switched decimal from treatment as string to treatment as decimal Added big decimal support for ElasticSearch Added big decimal support for hbase processors Added big decimal support for prometheus reporting Added big decimal support to solr Big decimal support in site to site reporting Core support for big decimal Fix issue with new toBigDecimal method Issue where method was not correctly handling the situation where a BigDecimal values was received as input. Update test to reflect use of BigDecimal Changes to unit tests to support DECIMAL type Fix bug where NaN Double reported from jvm metrics not handled BigDecimal implementation for mongo processor
|
Seem to have some issues reported from the Windows build. Not sure if they are related to something I have done or results of commits I've rebased from master. However I've pushed the changes as suggested above. Would appreciate you checking them again especially the hbase changes. Question. Should I look to add parameters to affected processors to allow users to select to turn on handling of Decimal type and try where not enabled to maintain current behaviour? |
|
As a rule of thumb, if the Linux build passes we can merge. Will try to find some time to review. |
MikeThomsen
left a comment
MikeThomsen
left a comment
There was a problem hiding this comment.
I added a few unit tests for DataTypeUtils and I'm not sure if my expectations are off or the patch isn't behaving correctly see examples:
public static final DecimalDataType DECIMAL_DATA_TYPE = new DecimalDataType(10, 5);
@Test
public void testStringToBigDecimal() {
String input = "10000.0005";
Object converted = DataTypeUtils.convertType(input, DECIMAL_DATA_TYPE, "testField");
assertNotNull(converted);
assertTrue(converted instanceof BigDecimal);
BigDecimal bd = (BigDecimal)converted;
assertEquals(10, bd.precision());
assertEquals(5, bd.scale());
}
@Test
public void testFloatToBigDecimal() {
float input = 100.0f;
Object converted = DataTypeUtils.convertType(input, DECIMAL_DATA_TYPE, "testField");
assertNotNull(converted);
assertTrue(converted instanceof BigDecimal);
BigDecimal bd = (BigDecimal)converted;
assertEquals(10, bd.precision());
assertEquals(5, bd.scale());
}
@Test
public void testDoubleToBigDecimal() {
double input = 100.0;
Object converted = DataTypeUtils.convertType(input, DECIMAL_DATA_TYPE, "testField");
assertNotNull(converted);
assertTrue(converted instanceof BigDecimal);
BigDecimal bd = (BigDecimal)converted;
assertEquals(10, bd.precision());
assertEquals(5, bd.scale());
}
@Test
public void testBigDecimalDatatypeSearch() {
Optional<DataType> matcher = DataTypeUtils.findMostSuitableType(
new BigDecimal("11.110"),
Arrays.asList(RecordFieldType.DECIMAL.getDecimalDataType(5, 3)),
Function.identity()
);
assertTrue(matcher.isPresent());
DataType dt = matcher.get();
assertTrue(dt instanceof DecimalDataType);
DecimalDataType ddt = (DecimalDataType)dt;
assertEquals(5, ddt.getPrecision());
assertEquals(3, ddt.getScale());
matcher = DataTypeUtils.findMostSuitableType(
new BigDecimal("1.110"),
Arrays.asList(RecordFieldType.DECIMAL.getDecimalDataType(9, 5)),
Function.identity()
);
assertTrue(matcher.isPresent());
ddt = (DecimalDataType)matcher.get();
assertEquals(9, ddt.getPrecision());
assertEquals(5, ddt.getScale());
}
Thoughts?
|
@markap14 could you take a look at the precision part and give me your opinion? |
|
@MikeThomsen Had a quick look at the unit tests and can see what you mean. Recall having a bit of debate with myself over how to handle this when initially implementing - there may be some scope for improvement. Basically we have two different areas of "precision & scale" A schema can define a type with precision and scale but then values manipulated from the records will also have a precision and scale. The DataType represented by DecimalDataType stores the schema defined precision and scale - but the values themselves, internally represented by the BigDecimal type take on the precision and scale of the value held. So a string of "100.00" assigned to a BigDecimal would have precision of 5 and scale of 2 I guess the question is do we want to enforce the schema defined precision and scale throughout the handling of values and where necessary truncate/round values so they adhere to the schema defined precision and scale? |
|
I'll try to find some time today to play around with the APIs to see what validation is even possible. The first thing that comes to mind is whether or not the Avro precision and scale values are not just advice versus hard limits that are validated. |
|
All the builds including windows should pass and failures need to be looked into. |
|
@joewitt ok, but we may need to reevaluate some of the tests because some of them have had timing issues on Windows that don't appear on a *NIX |
|
Please be specific about which tests on windows have timing issues. A ton of effort has gone into the tests on all platforms. A large number of tests were not written to work on windows but none of those that I've found are related to timing issues but instead are due to actual 'test wont work on windows' issues. The issue that occurred in this case appears to be a corrupt zip file. Reran the jobs |
|
Thanks Joe,
Will look into it. Previous commit had all jobs run fine, after rebasing the issues appeared. Understand that it’s better to be certain of cause.
Regards
Chris
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
…________________________________
From: Joe Witt <notifications@github.com>
Sent: Sunday, March 22, 2020 12:30:24 PM
To: apache/nifi <nifi@noreply.github.com>
Cc: Chris Manniex <cmanniex@hotmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [apache/nifi] NIFI-7159 - Add support for some data types in various processors (#4104)
Please be specific about which tests on windows have timing issues. A ton of effort has gone into the tests on all platforms. A large number of tests were not written to work on windows but none of those that I've found are related to timing issues but instead are due to actual 'test wont work on windows' issues. The issue that occurred in this case appears to be a corrupt zip file. Reran the jobs
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F4104%23issuecomment-602192509&data=02%7C01%7C%7Ce6d382ee92424a37566108d7ce5ccc03%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637204770255370792&sdata=Ln1DYRH%2FJ%2Fke%2FxRIqc6DKCQfz7U%2FLHXUfLBqpDeLUa0%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFHXE7HPE3UMHLTBTRV4WVLRIYAGBANCNFSM4K72OIGA&data=02%7C01%7C%7Ce6d382ee92424a37566108d7ce5ccc03%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637204770255380779&sdata=S1gvBDj%2FW%2BxPJuh6nZTl0cd1FL%2FNJ%2BZhML3TVkUd02c%3D&reserved=0>.
|
That's pretty much what I was getting at. |
|
We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. |
4 participants
Aligning the implementation of DECIMAL RecordFieldType with other types
Introduce RecordFieldType of DECIMAL for BigDecimal values
Adding RecordFieldType DECIMAL to ResultSetRecordSet
Introduce RecordFieldType DECIMAL into DataTypeUtils
Added missing BIGINT type to convertSimpleIfPossible method
Added missing type BIGINT to convertField method
Added missing type BIGINT to writeFieldForType
Added missing type BIGINT to parseFieldForType
Added missing type BIGINT to parseStringForType
Potentially breaking change of behaviour for Kudu Service.
Switched decimal from treatment as string to treatment as decimal
Added big decimal support for ElasticSearch
Added big decimal support for hbase processors
Added big decimal support for prometheus reporting
Added big decimal support to solr
Big decimal support in site to site reporting
Core support for big decimal
Fix issue with new toBigDecimal method
Issue where method was not correctly handling the situation where a
BigDecimal values was received as input.
Update test to reflect use of BigDecimal
Changes to unit tests to support DECIMAL type
Fix bug where NaN Double reported from jvm metrics not handled
BigDecimal implementation for mongo processor
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Enables X functionality; fixes bug NIFI-7159.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
[Y] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
[Y] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[Y] Has your PR been rebased against the latest commit within the target branch (typically
master)?[Y] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not
squashor use--forcewhen pushing to allow for clean monitoring of changes.For code changes:
mvn -Pcontrib-check clean installat the rootnififolder?LICENSEfile, including the mainLICENSEfile undernifi-assembly?NOTICEfile, including the mainNOTICEfile found undernifi-assembly?.displayNamein addition to .name (programmatic access) for each of the new properties?For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.