Skip to content

Real.realMod and Real.split produce incorrect result for values close to zero #269

Description

@pclayton

Version

110.99.3 (Latest)

Operating System

  • Any
  • Linux
  • macOS
  • Windows
  • Other Unix

OS Version

No response

Processor

  • Any
  • Arm (using Rosetta)
  • PowerPC
  • Sparc
  • x86 (32-bit)
  • x86-64 (64-bit)
  • Other

System Component

Basis Library

Severity

Minor

Description

For x : real where 0.0 < Real.abs x and Real.abs x <= 1E~16, both
Real.realMod x
#frac (Real.split x)
return 0.0 but should instead return x.

Transcript

- Standard ML of New Jersey (64-bit) v110.99.3 [built: Thu Sep 22 14:05:58 2022]
- val x = 1E~16;
val x = 1E~16 : real
- Real.realMod x;
[autoloading]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
val it = 0.0 : real
- Real.class it;
val it = ZERO : IEEEReal.float_class
- #frac (Real.split x);
val it = 0.0 : real
- Real.class it;
val it = ZERO : IEEEReal.float_class
- val x = ~1E~16;
val x = ~1E~16 : real
- Real.realMod x;
val it = 0.0 : real
- Real.class it;
val it = ZERO : IEEEReal.float_class
- #frac (Real.split x);
val it = 0.0 : real
- Real.class it;
val it = ZERO : IEEEReal.float_class

Expected Behavior

See description.

Steps to Reproduce

val x = 1E~16;
Real.realMod x;        (* should not be 0.0 *)
Real.class it;         (* should not be ZERO *)
#frac (Real.split x);  (* should not be 0.0 *)
Real.class it;         (* should not be ZERO *)

Additional Information

No response

Email address

phil.clayton@veonix.com

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingfixed-in-110.99.5issues that will be fixed in the 110.99.5 versionfloating-pointproblem related to floating-point operations

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions