Roadmap

[superbobry]

Could you sketch some of the future directions for owl?

I think OCaml community might benefit from a single numerical library and would like to contribute.

[ryanrhymes]

Thanks Sergei, this is a good suggestion. I will sketch out a future plan and specify some TODOs in the following days.

[ryanrhymes]

OK, here is a list of crude ideas: https://github.com/ryanrhymes/owl/wiki/Future-Plan

The plan can be adjusted based on the feedback, so all comments are welcome.

[nilsbecker]

hi, first of all it's great that you are addressing the lack of numerical computing infrastructure on ocaml!

the thing that i am missing most on the roadmap is something equivalent to the foundation of numpy: a multidimensional array that is a pleasure to manipulate, with slicing, (fancy) indexing, broadcasting, views, transpose, copy, whatnot. to my mind having that would be the start of tying the ocaml ecosystem together for numerics. matrices and matrix manipulation would then apply to two-dimensional (slices of) (sub)arrays.

are you planning on something like that? would the current design be compatible with such a type if someone builds something like that based on bigarray.genarray?

actually, i now saw that rleonid is indeed thinking about these thinks in his oml library. for the moment, it's based on float array array (at least partly) but it seems the idea is also to move to a general type. how do you the relation of oml to owl ?

[nilsbecker]

another thought: when i produce data with ocaml i often want to plot it using matplotlib (i think it's great that you are providing native plotting options but it's probably fair to say that that can't get as feature complete as mpl anytime soon) so it would be good to have easy to use export of matrices and/or vectors to a numpy-digestible format. i sometimes do memmapping and building a numpy array from the memmapped file but i find it quite cumbersome to handle.

or even to call out to python in some way (but that again seems like a big project)

[ryanrhymes]

N-dimensional array is a good suggestion and we definitely need that in owl. I will add it to the roadmap later. Thank you very much.

oml is a very cool library focusing more on stats (learning) if my understanding is correct, whereas Owl plans to target more general numerical operations (matrices, linear algebra, optimisation, and etc.). There is certain overlap at the moment but I guess they will diverge more at some point.

[ryanrhymes]

Matplotlib is already very mature but heavily integrated with Python. I think the best solution is to interface with Matplotlib with standard file format (as you suggested) which should be doable.

[nilsbecker]

hi, about the nd-array related work in oml i found this: rleonid/oml#166 . i guess the direction of oml is more statistics but they are forced to provide the array infrastructure to do that. i think there also was some talk about splitting off the actual statistics from the linear algebra in oml.

[ryanrhymes]

Cool, thanks. I read the thread you referred to and it seems we share a lot of common concerns. At this moment, I probably only focus on real and complex numbers.

Regarding the plotting, you can currently save matrices to csv files then import in numpy (although not very efficient comparing to marshal). I plan to use HDF5 in the future. But please do let me know if there are better (e.g., more efficient) solutions.