dart: Return dart:typed_data equivalent where possible

[NotTsunami]

Stems from discussion in #8183. This should allow users to be able to access a Uint8List directly without copying over from a list, thus actually offering zero-copy access (for this class, at least) in both the lazy path and the non-lazy path. This is pretty important for signal processing and image processing where we want to avoid any unnecessary list copies.

[jamesderlin]

ByteBuffer has an asInt8List method along with asUint8List, so for symmetry perhaps a Int8List _asInt8List(int offset, int length) method could be added Buffeto rContext, and _FbInt8List could be removed?

[NotTsunami]

ByteBuffer has an asInt8List method along with asUint8List, so for symmetry perhaps a Int8List _asInt8List(int offset, int length) method could be added Buffeto rContext, and _FbInt8List could be removed?

I'm not opposed to that, but I'd like to see some insight from one of the Dart maintainers to see if this aligns with their view for the Dart side. Ideally, we could expand the same to Uint16ListReader to use their more space-efficient dart:typed_data alternatives (assuming those two follow the same pattern of being subtypes of List<int>), but then the question arises should those classes also offer lazy/greedy paths?

[NotTsunami]

Ready for review. I tested using @tompark's example code, with the latest version of flatbuffers on pub.dev as well as this fork, and this fork prints the expected bundle1.image1 is already a Uint8List, len=29149 while the latest version on pub.dev does not. The only difference in my test path is that I manually executed the steps from setup.sh instead of writing a script. All tests are passing as well.

[NotTsunami]

I squashed the changes and removed a change related to my other PR to prevent conflicts between the two.

@vaind Hi Ivan, is there any way you might be able to spare some bandwidth in the relatively near future to help review and potentially land both of my open PRs? Cheers.

[vaind]

Thanks a lot. It looks pretty good to me, although there need to be some changes to make the code correct on Big endian machines (I know these are basically non-existent, but since Dart supports them, we should too).

[NotTsunami]

Thanks a lot. It looks pretty good to me, although there need to be some changes to make the code correct on Big endian machines (I know these are basically non-existent, but since Dart supports them, we should too).

I think I'd rather add a separate path or built-in for big-endian, otherwise the newly added conditions to the tests will cause the tests to fail on big-endian, as the old classes do not return Uint8List, Int8List, Uint16List, etc on the lazy path. I think the only supported test environment (theoretically) would be ARM on Linux running in big-endian mode? I'll try and set up a qemu environment to test.

[vaind]

Thanks a lot. It looks pretty good to me, although there need to be some changes to make the code correct on Big endian machines (I know these are basically non-existent, but since Dart supports them, we should too).

I think I'd rather add a separate path or built-in for big-endian, otherwise the newly added conditions to the tests will cause the tests to fail on big-endian, as the old classes do not return Uint8List, Int8List, Uint16List, etc on the lazy path. I think the only supported test environment (theoretically) would be ARM on Linux running in big-endian mode? I'll try and set up a qemu environment to test.

I think it's OK to have different actual return type on big endian, the only contract is that it is compatible with List, everything else is "just" an optimization. It's OK for tests to assert the return value is the expected type based on endianness though

[NotTsunami]

Thanks a lot. It looks pretty good to me, although there need to be some changes to make the code correct on Big endian machines (I know these are basically non-existent, but since Dart supports them, we should too).

I think I'd rather add a separate path or built-in for big-endian, otherwise the newly added conditions to the tests will cause the tests to fail on big-endian, as the old classes do not return Uint8List, Int8List, Uint16List, etc on the lazy path. I think the only supported test environment (theoretically) would be ARM on Linux running in big-endian mode? I'll try and set up a qemu environment to test.

I think it's OK to have different actual return type on big endian, the only contract is that it is compatible with List, everything else is "just" an optimization. It's OK for tests to assert the return value is the expected type based on endianness though

Sounds good to me. Sorry for the delay. I'll get this updated this week, hopefully on the earlier half as opposed to the later half.

[vaind]

LGTM.

Are you going to do the other lists (int32, ...) too or is this PR finished?

[NotTsunami]

LGTM.

Are you going to do the other lists (int32, ...) too or is this PR finished?

I'll extend this to other types! I'll expand the tests as well.

[NotTsunami]

@vaind I couldn't find out what is causing Float64List to fail, so I am going to revert the changes to the Float types and mark them with a todo to return their dart:typed_data equivalent so the PR can be restored to a working state and will be ready for review.

[vaind]

LGTM but I cannot merge as I'm not a maintainer.

[NotTsunami]

I had some time to sit down and review this again, and saw that the old code was causing potential misalignment in writeListFloat32 and writeListFloat64. Before we were treating all of the data as part of the count in _prepare, which technically does calculate the correct number of bytes but can cause misalignment, now we treat the length field correctly as a uint32 rather than as part of the float data, and specify the data as additionalBytes. I also let GH rebase this.

This is the final version of the PR, pending any code change requests from review. Hoping to see this actually go somewhere now,

[NotTsunami]

@aardappel Could you perhaps give this a look in spare time?

[aardappel]

@NotTsunami sorry, I have zero knowledge of Dart. I guess I am fine to merge if @vaind thinks it is ok.

@dbaileychess who probably know Dart better than I do.

Also, there is a failing test.

[NotTsunami]

@NotTsunami sorry, I have zero knowledge of Dart. I guess I am fine to merge if @vaind thinks it is ok.

@dbaileychess who probably know Dart better than I do.

Also, there is a failing test.

Thanks. Should be fixed. Pending review on the latest version of the PR now, then.

[sumitsharansatsangi]

Please merge it soon.