Skip to content

Incorrect rounding in f64_to_f16_fallback #17

@ivankra

Description

@ivankra

When comparing boa test262 results between amd64 vs arm64, I noticed there are some test failures are ultimately stem from a bug in f64_to_f16_fallback:

test/built-ins/DataView/prototype/setFloat16/set-values-return-undefined.js: Test262Error: value: 2.980232238769532e-8 Expected SameValue(«0», «5.960464477539063e-8») to be true
test/built-ins/Math/f16round/value-conversion.js: Test262Error: value: 2.980232238769532e-8 Expected SameValue(«0», «5.960464477539063e-8») to be true
test/built-ins/TypedArray/prototype/fill/fill-values-conversion-operations.js: Test262Error: 2.980232238769532e-8 converts to 5.960464477539063e-8 Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/built-ins/TypedArray/prototype/map/return-new-typedarray-conversion-operation.js: Test262Error: 2.980232238769532e-8 converts to 5.960464477539063e-8 Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/built-ins/TypedArray/prototype/set/array-arg-src-tonumber-value-conversions.js: Test262Error: [2.980232238769532e-8] => [5.960464477539063e-8] Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/built-ins/TypedArray/prototype/set/typedarray-arg-set-values-diff-buffer-other-type-conversions.js: Test262Error: Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/built-ins/TypedArrayConstructors/ctors/object-arg/conversion-operation.js: Test262Error: 2.980232238769532e-8 converts to 5.960464477539063e-8 Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/built-ins/TypedArrayConstructors/internals/DefineOwnProperty/conversion-operation.js: Test262Error: 2.980232238769532e-8 converts to 5.960464477539063e-8 Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/built-ins/TypedArrayConstructors/internals/Set/conversion-operation.js: Test262Error: The value of sample[0] is expected to equal the value of expected Expected SameValue(«0», «5.960464477539063e-8») to be true (Testing with Float16Array and makePassthrough.)
test/staging/sm/Math/f16round.js: Test262Error: Expected SameValue(«0», «5.960464477539063e-8») to be true

Test262's byteConversionValues.js defines these two constants that get misrounded by f64_to_f16_fallback:

    2.9802322387695312e-8,        // largest double which rounds to 0 when cast to float16
    2.980232238769532e-8,         // smallest double which does not round to 0 when cast to float16

If it helps, here's an algorithm used by V8 and another JavaScript engine in Rust that gets it right: math.rs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions