[zwave] encodeValue rejects valid BigDecimals; precision bug

[watou]

Below is a proposed change to the Z-Wave binding that would allow it to encode decimal numbers with any number of digits after the decimal point. The current code that converts BigDecimal numbers into the protocol format rejects them because of the number of digits, whereas it should only reject numbers that can't fit in 32 bits.

I don't know if a change I suggested to Chris before 1.8, where temperature unit conversion was being done inexactly using native floating point but could be done exactly with BigDecimal, helped avoid the excessive strictness in the code below, or if these ArithmeticExceptions were being thrown before 1.8.0. I suspect they were. Either way, the 1.8.0 version of the encodeValue method throws ArithmeticExceptions under incorrect conditions, whereas the proposed change below will accept all numbers that can fit in 32 bits, but may apply rounding in order to accommodate the encoding format.

This change also fixes a bug where trying to encode 0.12345678 will send 12345678!

Proposed change:

--- a/bundles/binding/org.openhab.binding.zwave/src/main/java/org/openhab/binding/zwave/internal/protocol/commandclass/ZWaveCommandClass.java
+++ b/bundles/binding/org.openhab.binding.zwave/src/main/java/org/openhab/binding/zwave/internal/protocol/commandclass/ZWaveCommandClass.java
@@ -11,7 +11,7 @@ package org.openhab.binding.zwave.internal.protocol.commandclass;
 import java.lang.reflect.Constructor;
 import java.lang.reflect.InvocationTargetException;
 import java.math.BigDecimal;
-import java.math.BigInteger;
+import java.math.MathContext;
 import java.util.HashMap;
 import java.util.Map;
 import java.lang.NumberFormatException;
@@ -46,7 +46,9 @@ public abstract class ZWaveCommandClass {
 //     private static final SCALE_SHIFT = 0x03; // unused
        private static final int PRECISION_MASK = 0xe0;
        private static final int PRECISION_SHIFT = 0x05;
-       
+       private static final int MAX_PRECISION = 7;
+       private static final MathContext PRECISION_CONTEXT = new MathContext(MAX_PRECISION);
+
        @XStreamOmitField
        private ZWaveNode node;
        @XStreamOmitField
@@ -314,6 +316,12 @@ public abstract class ZWaveCommandClass {
         */
        protected byte[] encodeValue(BigDecimal value) throws ArithmeticException {

+           // Throw ArithmeticException if value is out of range
+           if (value.compareTo(BigDecimal.valueOf(Integer.MAX_VALUE)) > 0 ||
+            value.compareTo(BigDecimal.valueOf(Integer.MIN_VALUE)) < 0) {
+            throw new ArithmeticException("value is out of range");
+        }
+
                // Remove any trailing zero's so we send the least amount of bytes possible
                BigDecimal normalizedValue = value.stripTrailingZeros();

@@ -324,10 +332,9 @@ public abstract class ZWaveCommandClass {
                        normalizedValue = normalizedValue.setScale(0);
                }

-               if (normalizedValue.unscaledValue().compareTo(BigInteger.valueOf(Integer.MAX_VALUE)) > 0) {
-                       throw new ArithmeticException();
-               } else if (normalizedValue.unscaledValue().compareTo(BigInteger.valueOf(Integer.MIN_VALUE)) < 0)
-                       throw new ArithmeticException();
+               if (normalizedValue.scale() > MAX_PRECISION) {
+                       normalizedValue = normalizedValue.round(PRECISION_CONTEXT);
+               }

                // default size = 4
                int size = 4;

Current behaviour:

About to encode '0'
Normalized value is 0, scale=0
0100

About to encode '22.111111111111110716365146799944341182708740234375'
java.lang.ArithmeticException <<< INCORRECT: should round instead of reject

About to encode '22.123'
Normalized value is 22.123, scale=3
62566b

About to encode '22.12345'
Normalized value is 22.12345, scale=5
a40021c1f9

About to encode '0.12345678'
Normalized value is 0.12345678, scale=8   <<< BUG in current code: scale must fit in 3 bits
0400bc614e

About to encode '223456.12345678'
java.lang.ArithmeticException  <<< INCORRECT: should round instead of reject

About to encode '-223456.12345678'
java.lang.ArithmeticException <<< INCORRECT: should round instead of reject

About to encode '2234567.123456745034598340983405983408'
java.lang.ArithmeticException <<< INCORRECT: should round instead of reject

About to encode '1234567890.123456745034598340983405983408'
java.lang.ArithmeticException <<< INCORRECT: should round instead of reject

About to encode '1234567890'
Normalized value is 1234567890, scale=0
04499602d2

About to encode '223456712345674503459.8340983405983408'
java.lang.ArithmeticException

Behaviour of proposed change:

About to encode '0'
Normalized value is 0, scale=0
0100

About to encode '22.111111111111110716365146799944341182708740234375'
Normalized value is 22.11111, scale=5
a40021bd27

About to encode '22.123'
Normalized value is 22.123, scale=3
62566b

About to encode '22.12345'
Normalized value is 22.12345, scale=5
a40021c1f9

About to encode '0.12345678'
Normalized value is 0.1234568, scale=7
e40012d688

About to encode '223456.12345678'
Normalized value is 223456.1, scale=1
24002218c1

About to encode '-223456.12345678'
Normalized value is -223456.1, scale=1
24ffdde73f

About to encode '2234567.123456745034598340983405983408'
Normalized value is 2234567, scale=0
04002218c7

About to encode '1234567890.123456745034598340983405983408'
Normalized value is 1234568000, scale=0
0449960340

About to encode '1234567890'
Normalized value is 1234567890, scale=0
04499602d2

About to encode '223456712345674503459.8340983405983408'
java.lang.ArithmeticException: value is out of range

[watou]

This change also fixes a bug where trying to encode 0.12345678 will send 12345678!