Use const 8 byteWidth for Timestamp and Duration

[TheMrCam]

Overview

Should resolve #129 by setting the byteWidth correctly (according to the advice in the issue) to 8.

Details

I realized that getTimeByteWidth was only being used by Timestamp and Duration, so I ended up just removing it and hardcoding the byteWidth.

I also wasn't able to successfully run the tests, but I did successfully run yarn build (with ArrayBufferLike warnings). I can run the tests if necessary, but I may need a little guidance on the rust-arrow-ffi setup.

[kylebarron]

It would be great to have tests specifically for the time, duration, and timestamp array types.

I think you just need to update the version of wasm-bindgen for the tests to run.

[kylebarron]

It looks like #131 at least gets past that error point

[kylebarron]

You can try merging in main and see if that fixes your tests

[TheMrCam]

I merged in main, and went through and got yarn test working locally. I did add wasm-pack to rust-arrow-ffi's devDependencies, so I pushed that and the yarn.lock changes, but I can revert if those updates are unnecessary or unwanted.

I see the yarn.lock changes broke the CI while I was writing this. Should I force push a reset without that latest commit?

[kylebarron]

I think adding wasm-pack to dev dependencies is fine.

Is there a reason why yarn install locally wouldn't create a yarn.lock that isn't consistent with the version in CI? Do we need to do something to match the yarn versions?

[TheMrCam]

I assume the yarn.lock discrepancy is due to my yarn version: 4.9.4. I normally use npm, so I had to install yarn to use the yarn.lock rather than create a package-lock.json. It looks like the rust-arrow-ffi lockfile version went from 6 to 8, so I'd guess that one was generated with an older yarn version. Interestingly, the top level yarn.lock was already version 8.

I can figure out which yarn version corresponds to a version 6 yarn.lock and re-run yarn install within rust-arrow-ffi. Unless the lockfile version change is because there were no dependencies before, but I added wasm-pack? I'll look into that too.

[kylebarron]

You can bump the version of yarn here:

arrow-js-ffi/package.json

Line 54 in e03a75c

"yarn": "4.0.2"

which should get picked up as the version of yarn used by CI:

arrow-js-ffi/.github/workflows/test.yml

Lines 19 to 20 in e03a75c

	- name: Set up Volta
	uses: volta-cli/action@v4

[kylebarron]

Would you be able to try uncommenting this test?

arrow-js-ffi/tests/ffi.test.ts

Lines 594 to 619 in e03a75c

	// // This also looks to be failing; probably an issue with the byte width?
	// test.skip("timestamp", (t) => {
	// let columnIndex = TEST_TABLE.schema.fields.findIndex(
	// (field) => field.name == "timestamp"
	// );
	// const originalField = TEST_TABLE.schema.fields[columnIndex];
	// const originalVector = TEST_TABLE.getChildAt(columnIndex)!;
	// const fieldPtr = FFI_TABLE.schemaAddr(columnIndex);
	// const field = parseField(WASM_MEMORY.buffer, fieldPtr);
	// expect(field.name).toStrictEqual(originalField.name);
	// expect(field.typeId).toStrictEqual(originalField.typeId);
	// expect(field.nullable).toStrictEqual(originalField.nullable);
	// const arrayPtr = FFI_TABLE.arrayAddr(0, columnIndex);
	// const wasmVector = parseVector(WASM_MEMORY.buffer, arrayPtr, field.type);
	// console.log(wasmVector.toJSON());
	// for (let i = 0; i < 3; i++) {
	// console.log(originalVector.get(i));
	// console.log(wasmVector.get(i));
	// expect(originalVector.get(i), wasmVector.get(i));
	// }
	// });

It should be able to load this timestamp array defined in python:

arrow-js-ffi/tests/pyarrow_generate_data.py

Lines 108 to 117 in e03a75c

	def timestamp_array() -> pa.Array:
	arr = pa.TimestampArray.from_pandas(
	[datetime.now(), datetime.now(), datetime.now()],
	type=pa.timestamp("s", tz="America/New_York"),
	)
	assert isinstance(arr, pa.TimestampArray)
	assert arr.type.unit == "s"
	assert arr.type.tz == "America/New_York"
	return arr

[kylebarron]

Awesome, thanks!