Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bigarray from string #8519

Closed
codename68 opened this Issue Mar 18, 2019 · 5 comments

Comments

Projects
None yet
4 participants
@codename68
Copy link

codename68 commented Mar 18, 2019

I'd like to file a feature request about adding a new function that takes:
string -> ('a, 'b) Bigarray.kind -> 'c Bigarray.layout -> int array -> ('a, 'b, 'c) Bigarray.Genarray.t
It would interpret the string's contents as contents that came from memory.

Let me just talk about the use case I've had and why I would like that.
I've tried to write a program that takes as input a dynamic array from a fifo (the type information and the length of the array are given first on the fifo, and then the array data).
The perfect candidate to store the array contents is Bigarray, it has already a perfect kind type to represent the type information.
The only function in the standard library that is able to return a Bigarray from raw data, as of now and unless I'm mistaken, is Unix.map_file.
This function is, unfortunately, not able to work on fifos, so I need to first read the data into memory, and then somehow transform that string into a bigarray.
This is impossible with the functions presently available, at least without resorting to writing that string to a file and then use Unix.map_file, which, you'll convene, is not particularly beautiful.

@nojb

This comment has been minimized.

Copy link
Contributor

nojb commented Mar 18, 2019

Off the top of my head, several libraries that carry a function to convert a string to a bigarray:

@codename68

This comment has been minimized.

Copy link
Author

codename68 commented Mar 19, 2019

Hi,

none of these libraries permit to select the kind and dimensions of your data.

Maybe it would be more clear if I say that I want a function that takes a Bytes.t instead of string: I'm not interested into the character array in itself, but as a representation of underlying data.

@yallop

This comment has been minimized.

Copy link
Member

yallop commented Mar 19, 2019

I'd like to file a feature request about adding a new function that takes:
string -> ('a, 'b) Bigarray.kind -> 'c Bigarray.layout -> int array -> ('a, 'b, 'c) Bigarray.Genarray.t

The memcpy package makes it fairly easy to write such a function:

open Bigarray
let bacopy : type a b. bytes -> (a, b) kind -> int array -> (a, b, c_layout) Genarray.t =
  fun src k dims ->
  let dst = Genarray.create k c_layout dims in
  Memcpy.(memcpy_from_bytes (bigarray Ctypes.genarray dims k)) ~src ~dst ~dst_off:0;
  dst

There's currently no support for Fortran-layout arrays, but it wouldn't be hard to add.

@codename68

This comment has been minimized.

Copy link
Author

codename68 commented Mar 19, 2019

Thanks!
This is what I need.

I still think that this functionality should be present somewhere in the stdlib, as a more generic Unix.map_file; and the dependency to ctypes just to be able to reinterpret data is big.

@xavierleroy

This comment has been minimized.

Copy link
Contributor

xavierleroy commented Mar 21, 2019

I think the need is uncommon enough that it's OK to require an external library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.