implement int64,uint64,int,uint for JS backend

[vegansk]

We need the implementation of int64/uint64 types for js backend. Now it's broken for int64 and absent for uint64.

For example, here is how they do it in ScalaJS: https://github.com/scala-js/scala-js/blob/10d67c9e479b40714a9e134522a181829b8f64bc/library/src/main/scala/scala/scalajs/runtime/RuntimeLong.scala

[vegansk]

Problem 1:

template ordLimitDecl(typ: typedesc): untyped =
  proc min(t = typ): typ = low(typ)
  proc max(t = typ): typ = high(typ)

int64.ordLimitDecl

echo int64.min

C version prints -9223372036854775808, js: -9223372036854776000.

[vegansk]

Problem 2:

let x = 0'u64

echo x

Produces:

p2.nim(1, 9) Error: internal error: gen: unknown node type: nkUInt64Lit
No stack traceback available
To create a stacktrace, rerun compilation with ./koch temp js <file>

[vegansk]

Problem 3:

let x = 0'u32

echo x

Produces:

p2.nim(3, 6) Error: type mismatch: got (uint32)
but expected one of: 
proc `$`(x: string): string
proc `$`(x: int): string
proc `$`(x: char): string
proc `$`[Enum: enum](x: Enum): string
proc `$`[T](x: set[T]): string
proc `$`(x: float): string
proc `$`(x: int64): string
proc `$`[T](x: seq[T]): string
proc `$`[T: tuple |
    object](x: T): string
proc `$`(x: bool): string
proc `$`(x: cstring): string

[vegansk]

Problem 4:

import parseutils

var x: int64
discard parseBiggestInt($int64.high, x)
echo x

Fails at runtime:

/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:403
    throw new Error(cbuf_12801);
          ^
Error: Error: unhandled exception: over- or underflow [OverflowError]
Traceback (most recent call last)
parseutils.parseBiggestInt, line: 237
parseutils.rawParseInt, line: 222

    at unhandledException (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:403:11)
    at raiseException (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:216:1)
    at raiseOverflow (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:414:1)
    at mulInt (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:252:56)
    at rawparseint_25957 (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:458:31)
    at npuParseBiggestInt (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:486:16)
    at Object.<anonymous> (/home/vega/work/nim/nimtests/issues/4714/nimcache/p4.js:492:1)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)

[Araq]

Yes, please fix these. :-)

[philip-wernersbach]

Part of the issue here is that the Javascript standard mandates that all Numbers in Javascript are represented as IEEE-754 floating point numbers. Which means that you get 53 bits of precision (Number.MAX_SAFE_INTEGER + 1), and then after a number becomes larger than that, the engine will start representing the number in approximations. So that's why int64 is broken. uint64 would have the same issue.

So as far as problem 1, max(int64) and min(int64) on the JS platform should take this limitation into account and evaluate to the limits of the Javascript engine's absolute precision. There could also be a flag "-d:emulate64bitNumbersSlow", which could force all 64 bit integers to use a software big number implementation supplied by the compiler.

The other problems are valid :)

[markus-oberhumer]

I think this is a design problem which really must be fixed before Nim 1.0.

Other compilers/transpilers that target JS like GopherJS and Kotlin get this right by using runtime functions for proper 64-bit handling.

[Araq]

So copy their solution.

[skilchen]

This is an attempt to copy the kotlin solution. Kotlin only has code for signed int64. I don't know how something like this could be better integrated in Nim.

[Araq]

Not an issue for v1. You can use emscripten for low level code and high level code is about targeting the DOM which in JS and has no need for int64 and friends.

[timotheecour]

#16411 was closed as duplicate for this issue so I'm adding its test case here:

problem 5

echo uint64.high

js: -1
c: 18446744073709551615

[konsumlamm]

This issue is currently blocking JS support for nim-lang/bigints, see nim-lang/bigints#59.

[sjrd]

If someone is inclined to integrate the Scala.js implementation, I can provide guidance about it, having written it. (I know nothing about Nim nor its compiler, though.) To the best of our knowledge, the Scala.js implementation is the fastest implementation of 64 bit integers for JavaScript. It also features the unsigned operations (unsigned division, modulo, toString and fromString; the others are the same whether they're signed or not).

The latest version is here:
https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/linker-private-library/src/main/scala/org/scalajs/linker/runtime/RuntimeLong.scala
and parsing from string here:
https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/javalanglib/src/main/scala/java/lang/Long.scala#L190-L317

By the way, we also have correct implementations of float32 operations, in case you're interested.

[ringabout]

@sjrd Thanks for your help! Your implementation looks great.

By the way, we also have correct implementations of float32 operations, in case you're interested.

May I have the link?

[sjrd]

By the way, we also have correct implementations of float32 operations, in case you're interested.

May I have the link?

Sure. It's a bit more "all over the place", but here are the relevant moving parts: (in Scala, Float refers to float32, and Double refers to float64)

• Arithmetic operations + - * / % have simple translations, which only require a Math.fround, e.g., a + b becomes Math.fround(a + b) when the operands are Floats.
• The double-to-float conversion is by definition Math.fround; the intXX-to-float conversions for XX <= 32 are implemented by first converting to Double (which is lossless), then applying Math.fround.
• The long-to-float conversion is surprisingly tricky. The naive long-to-double-to-float path is incorrect. Our implementation is at https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/linker-private-library/src/main/scala/org/scalajs/linker/runtime/RuntimeLong.scala#L609-L662
• Parsing is super tricky. The naive string-to-double-to-float is incorrect (where string-to-double is JavaScript's parseFloat). Our implementation is at https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/javalanglib/src/main/scala/java/lang/Float.scala#L85-L314, with the hexadecimal parsing shared with that of Double at https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/javalanglib/src/main/scala/java/lang/Double.scala#L140-L273
• If you target ECMAScript < 2015 (typically, ES 5.1), you'll need a polyfill of Math.fround when it's not available. Our implementation is a JS AST directly encoded in Scala at https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/linker/shared/src/main/scala/org/scalajs/linker/backend/emitter/CoreJSLib.scala#L199-L301, but you can also see the JavaScript code that it generates at https://gist.github.com/sjrd/9b685abad5184d26cc0670fb25bef7fe

As for tests, here are links as well:

• https://github.com/scala-js/scala-js/blob/main/test-suite/shared/src/test/scala/org/scalajs/testsuite/javalib/lang/FloatTest.scala (notably parsing, and formatting to hex string)
• https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/test-suite/shared/src/test/scala/org/scalajs/testsuite/compiler/DoubleTest.scala#L61-L204 for the double-to-float conversion (which basically tests the fround polyfill, when necessary)

You may also be interested in our polyfills to read/write the bits of float{32,64} to int{32,64}, here:
https://github.com/scala-js/scala-js/blob/f115c4786579523ff9e97eceaf834dd3da15aa45/javalanglib/src/main/scala/java/lang/FloatingPointBits.scala#L154

That should cover the essentials. If there's anything else you'd like to know, feel free to ask. ;)

[ringabout]

Thanks for the elaborate explanation. I will keep these in mind.