Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JS] DateMilliseconds is not always decoded correcly #40718

Closed
Fil opened this issue Mar 21, 2024 · 4 comments
Closed

[JS] DateMilliseconds is not always decoded correcly #40718

Fil opened this issue Mar 21, 2024 · 4 comments

Comments

@Fil
Copy link

Fil commented Mar 21, 2024

Doing a round-trip with a JavaScript Date in apache-arrow/js sometimes results in wrong values:

import * as Arrow from "apache-arrow";
const date = new Date(1950, 1, 0);
const data = [{date}]
console.log("true date", date, +date);
const arrow = Arrow.tableFromJSON(data);
const [lo, hi] = arrow.batches[0].data.children[0].values;
console.log("encoded bytes", hi * (2**32) + lo);
for (const d of arrow) console.log("decoded date", d.date, +d.date)

this prints:

❯ node test-arrow.js
true date 1950-01-30T23:00:00.000Z -628563600000
encoded bytes -628563600000
decoded date 1950-03-21T16:02:47.296Z -624268632704

As you can see there is a difference of 45 days. If I go and modify factories.ts and change dtypes.DateMillisecond to dtypes.Float64 (line 122), the decoding returns the correct number.

version information:

apache-arrow@^15.0.2:
  resolved "https://registry.yarnpkg.com/apache-arrow/-/apache-arrow-15.0.2.tgz#d87c6447d64d6fab34aa70119362680b6617ce63"

❯ node -v
v20.11.1

macos sonoma 14.3.1 (23D60)

The results are correct for dates after epoch, but testing with new Date(1969, 11, 31); exhibits the problem.

Component(s)

JavaScript

@trxcllnt
Copy link
Contributor

Looks like these lines should be Math.floor() instead of Math.trunc().

@Fil
Copy link
Author

Fil commented Mar 21, 2024

Yes that works for me! thanks

domoritz added a commit to domoritz/arrow that referenced this issue Mar 21, 2024
@domoritz
Copy link
Member

Fix in #40725 and I added a simple test.

@kou kou changed the title in arrow.js, DateMilliseconds is not always decoded correcly [JS] DateMilliseconds is not always decoded correcly Mar 21, 2024
domoritz added a commit to domoritz/arrow that referenced this issue Mar 22, 2024
domoritz added a commit to domoritz/arrow that referenced this issue Mar 22, 2024
@domoritz
Copy link
Member

Fixed in #40725

@domoritz domoritz added this to the 16.0.0 milestone Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants