Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix various issues with handling of floating values within the LWM2M subsystem #16154

Closed
lodup29 opened this issue May 14, 2019 · 4 comments · Fixed by #18716
Closed

Fix various issues with handling of floating values within the LWM2M subsystem #16154

lodup29 opened this issue May 14, 2019 · 4 comments · Fixed by #18716
Assignees
Labels
area: LWM2M bug The issue is a bug, or the PR is fixing a bug priority: medium Medium impact/importance bug

Comments

@lodup29
Copy link
Contributor

lodup29 commented May 14, 2019

Describe the bug

  • Regardless of its representation (TLV, JSON), it's not possible to pass a value > -1.0 and < 0.0 (ex: -0.5). There's no sign field in the float32_value and float64_value so the only way to represent a value such as -0.5 is :
struct float32_value lFloatValue; /* val1  and val2 are integers (s32_t) */
lFloatValue.val1 = 0;
lFloatValue.val2 = -500000;

However only the sign of val1 is considered for TLV while the JSON representation simply formats the value as ("%d.%d",lFloatValue.val1, lFloatValue.val2) ("0.-500000") thus resulting in an exception when attempting to parse the value on the other side.

  • Using the JSON representation it is not possible to represent a value with leading zeroes for val2 (ex: 0.05). Indeed as such value should be reprensented as:
struct float32_value lFloatValue; /* val1  and val2 are integers (s32_t) */
lFloatValue.val1 = 0;
lFloatValue.val2 = 50000; /* 0.05*1000000 */

However as previously stated the value will simply end up being formated as ("%d.%d",lFloatValue.val1, lFloatValue.val2) ("0.500000" instead of "0.050000")

  • At least some mantissas are one bit too short using the TLV reprensentation. Trying the represent 0.05 (0x3D4CCCCD using IEEE 754 single precision) ends up as 0.049999997 (0x3D4CCCCC) on the other side

To Reproduce

  • Use the Leshan lwm2m server demo as the server backend
wget https://hudson.eclipse.org/leshan/job/leshan/lastSuccessfulBuild/artifact/leshan-server-demo.jar
sudo java -jar ./leshan-server-demo.jar -wp 80
  • Select Single-value in the top right drop-down menu as JSON or TLV based on the test case
  • Use the zephyr lwm2m client example and change the value of the temperature object as described by the following cases:
Representation val1 val2 Expected value Value retrieved by Leshan
TLV 0 -500000 -0.5 0.5
JSON 0 -500000 -0.5 Exception
JSON 0 50000 0.05 0.5
TLV 0 50000 0.05 0.049999997
@lodup29 lodup29 added the bug The issue is a bug, or the PR is fixing a bug label May 14, 2019
@lodup29 lodup29 changed the title Fix various issues with handling of floating value within the LWM2M subsystem Fix various issues with handling of floating values within the LWM2M subsystem May 14, 2019
@nashif nashif added the priority: medium Medium impact/importance bug label May 21, 2019
@ioannisg
Copy link
Member

@mike-scott, same question as before; wonder if this is still applicable, and, if so, if it could be addressed for 2.0

@mike-scott
Copy link
Contributor

Thank you for the reminder. I will address.

@ioannisg
Copy link
Member

Thank you for the reminder. I will address.

Just asking if this has this already been addressed, @mike-scott - few LWM2M Pr were merged lately..

@mike-scott
Copy link
Contributor

mike-scott commented Aug 27, 2019

@lodup29 @ioannisg I've setup a PR which addresses the first 3 items (and a few other bugs), however item 4 in the table above:

Representation val1 val2 Expected value Value retrieved by Leshan
TLV 0 50000 0.05 0.049999997

Is actually not a bug, but reflects a conversion issue going from fixed formatting to binary float format. This can be reproduced using the following website:
https://www.exploringbinary.com/binary-converter/

Enter "0.05" in the Decimal to Binary section at the top, and then click "Convert"
Resulting binary value is:
0.00001100110011001100110011001100110011001100110011001100110011 (repeating)

Copy that binary value into the Binary to Decimal section below and then click "Convert"
Resulting decimal value is:
0.04999999999999999995663191310057982263970188796520233154296875

NOTE: Tool truncates infinite binary fractions to 62 bits. I'm using less than that so value is truncated earlier.

mike-scott added a commit to mike-scott/zephyr that referenced this issue Aug 27, 2019
Formatting a float32/64 value for plain text is broken.
Example for 32bit: val1=0 and val2=500000 is equivalent to 0.5

Current formatter was using %d.%d (%lld.%lld for 64bit) so
exported value was 0.500000 (or 0.5)

To fix this, for val2 use a zero-padded formatter for the maximum
length of each bit length (6 for 32bit and 9 for 64bit), and then
remove the zero characters at the end of the string.

Notes re: handling of val1/val2 signs:
- eliminate potential negative sign when converting val2 to avoid:
  a value like: 0.-5
- use negative val2 when val1 is 0 to fix small negative handling
  such as -0.5

Fixes: zephyrproject-rtos#16154

Signed-off-by: Michael Scott <mike@foundries.io>
mike-scott added a commit to mike-scott/zephyr that referenced this issue Aug 27, 2019
Current JSON formatting for float32/64 is broken in a similar way as
plain text.  Let's use the newly fixed logic for plain text to
generate the float32/64 values in the JSON string.

Fixes: zephyrproject-rtos#16154

Signed-off-by: Michael Scott <mike@foundries.io>
mike-scott added a commit to mike-scott/zephyr that referenced this issue Aug 27, 2019
When val1 is 0, we need to handle a negative val2 value so that we
generate correct TLV value.

Example: val1 = 0, val2 = -500000 is equivalent to -0.5 decimal.
Currently we generate: 0.5 (losing the sign).

Fixes: zephyrproject-rtos#16154

Signed-off-by: Michael Scott <mike@foundries.io>
ioannisg pushed a commit that referenced this issue Aug 28, 2019
Formatting a float32/64 value for plain text is broken.
Example for 32bit: val1=0 and val2=500000 is equivalent to 0.5

Current formatter was using %d.%d (%lld.%lld for 64bit) so
exported value was 0.500000 (or 0.5)

To fix this, for val2 use a zero-padded formatter for the maximum
length of each bit length (6 for 32bit and 9 for 64bit), and then
remove the zero characters at the end of the string.

Notes re: handling of val1/val2 signs:
- eliminate potential negative sign when converting val2 to avoid:
  a value like: 0.-5
- use negative val2 when val1 is 0 to fix small negative handling
  such as -0.5

Fixes: #16154

Signed-off-by: Michael Scott <mike@foundries.io>
ioannisg pushed a commit that referenced this issue Aug 28, 2019
Current JSON formatting for float32/64 is broken in a similar way as
plain text.  Let's use the newly fixed logic for plain text to
generate the float32/64 values in the JSON string.

Fixes: #16154

Signed-off-by: Michael Scott <mike@foundries.io>
ioannisg pushed a commit that referenced this issue Aug 28, 2019
When val1 is 0, we need to handle a negative val2 value so that we
generate correct TLV value.

Example: val1 = 0, val2 = -500000 is equivalent to -0.5 decimal.
Currently we generate: 0.5 (losing the sign).

Fixes: #16154

Signed-off-by: Michael Scott <mike@foundries.io>
LeiW000 pushed a commit to LeiW000/zephyr that referenced this issue Sep 2, 2019
Formatting a float32/64 value for plain text is broken.
Example for 32bit: val1=0 and val2=500000 is equivalent to 0.5

Current formatter was using %d.%d (%lld.%lld for 64bit) so
exported value was 0.500000 (or 0.5)

To fix this, for val2 use a zero-padded formatter for the maximum
length of each bit length (6 for 32bit and 9 for 64bit), and then
remove the zero characters at the end of the string.

Notes re: handling of val1/val2 signs:
- eliminate potential negative sign when converting val2 to avoid:
  a value like: 0.-5
- use negative val2 when val1 is 0 to fix small negative handling
  such as -0.5

Fixes: zephyrproject-rtos#16154

Signed-off-by: Michael Scott <mike@foundries.io>
LeiW000 pushed a commit to LeiW000/zephyr that referenced this issue Sep 2, 2019
Current JSON formatting for float32/64 is broken in a similar way as
plain text.  Let's use the newly fixed logic for plain text to
generate the float32/64 values in the JSON string.

Fixes: zephyrproject-rtos#16154

Signed-off-by: Michael Scott <mike@foundries.io>
LeiW000 pushed a commit to LeiW000/zephyr that referenced this issue Sep 2, 2019
When val1 is 0, we need to handle a negative val2 value so that we
generate correct TLV value.

Example: val1 = 0, val2 = -500000 is equivalent to -0.5 decimal.
Currently we generate: 0.5 (losing the sign).

Fixes: zephyrproject-rtos#16154

Signed-off-by: Michael Scott <mike@foundries.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: LWM2M bug The issue is a bug, or the PR is fixing a bug priority: medium Medium impact/importance bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants