Don't transfer input ArrayBuffers

[reillyeon]

Right now the input and output ArrayBuffers passed to MLContext's compute() method are transferred so that their contents cannot be modified by script during the computation. It seems appropriate that an ArrayBuffer passed to MLGraphBuilder's constant() method should be treated the same way, though it would be better if the transfer were delayed until build() is called so that multiple ArrayBufferViews using the same backing ArrayBuffer can be passed as constants. build() would construct a list of unique backing ArrayBuffers to transfer rather than the naive approach of simply iterating over the ArrayBufferViews themselves.

I'll note that transferring ArrayBuffer parameters makes an API harder to call from WebAssembly, but I think that any argument for why compute inputs and outputs are transferred should also apply to constants.

[huningxin]

@reillyeon

but I think that any argument for why compute inputs and outputs are transferred should also apply to constants.

Agreed.

I'll note that transferring ArrayBuffer parameters makes an API harder to call from WebAssembly

This is true.

Should we consider copying the content of input (and constant) ArrayBuffer instead? Notice that the Chromium implementation actually copies the input and constant ArrayBuffers for XNNPACK backend and DirectML (mojo) backend. It means for each input ArrayBuffer, WebAssembly path may need two copies, one is done by JS that copies from Wasm heap to an ArrayBuffer that can be transferred, another one is done by Chromium implementation that copies from ArrayBuffer to internal storage.

[reillyeon]

Given the constraints on implementations based on the underlying platform APIs I think it's reasonable to define constant and input handling such that MLGraphBuilder and compute() copy any provided ArrayBuffers. If developers want to avoid a copy they can use MLBuffers instead.

In theory Chromium's XNNPACK backend could avoid copies but I'm also planning to delete it in favor of a more flexible TFLite backend. The zero-copy option only works for CPU backends anyways (without MLBuffer).

[reillyeon]

Renamed this issue given my current position on how to resolve the inconsistency between handling of constant and input ArrayBuffers.