Confused about types

[bluddy]

I think this should work:

open Torch
Tensor.(arange1 ~start:(f 0.) ~end:(f 1.))

But I get

Error: This expression has type Tensor.t but an expression was expected of type
         Torch_core.Wrapper.Scalar.t

[LaurentMazare]

PyTorch has a separate internal representation for scalars (0 dimension tensors) and tensors - the ocaml types for these are respectively Scalar.t and Tensor.t. You should be able to use the arange1 function via:

 Tensor.arange1 ~start:(Scalar.int 0) ~end_:(Scalar.int 5) ~options:(Int, Cpu);;

Let me know if you have any further issue using this.

[bluddy]

I see. So what does f do in Tensor? And if it's a value, wouldn't it make sense to mirror that interface in Scalar?

Also, would it be possible to put up the odoc documentation on a github.io page?

[bluddy]

Also:

• Using first and last rather than begin and end makes more sense just because end isn't available and remembering the underscore is a pain.
• It seems like arange could just have step as an optional argument?

And this library needs documentation. But I'm sure you know that :)

[bluddy]

Sorry for bombarding with questions here, but I don't know where to do so. Feel free to drop by the ocaml discord if you want to discuss more, or if you even want a channel for discussing this library.

How would one iterate over a tensor the same way as you do in pytorch? I feel like there should be a Tensor.iter function?

[LaurentMazare]

• Tensor.f creates a tensor with no dimensions which is very similar to a scalar but the internal PyTorch representations are different, this would be mostly used for broadcasting operations, e.g. doing things like Tensor.(t + f 42.). Having Scalar.f and Scalar.i on top of Scalar.float and Scalar.int would sound reasonable to me.
• The tensor api is mostly generated automatically from a yaml file coming from the PyTorch distribution Declarations.yaml, e.g. arange1 is defined in this file. I used an integer suffix (e.g. arange1, arange2) because the C++ PyTorch api uses overloading which does not have an equivalent in ocaml.
• Currently Declarations.yaml does not contain any documentation, there is some project to clean it up, hopefully this will allow having better types on the ocaml side Declarations.yaml cleanup pytorch/pytorch#12562 but I don't think adding documentation there is a part of this project.
• As the api is automatically generated, the parameter names match the python ones hence it uses begin and end rather than first and last. Maybe would it make sense to have a more 'ocaml' api on top but I think it's nice to also have this low level api as close as possible to the python api.
• step is available if you use arange2 rather than arange1 (again not having overloading is not ideal here).
• I don't think one would iterate over a PyTorch tensor when using the python api (the tensor may be on a gpu anyway). If you need to do something like this you should probably convert it to a list/array first - but most of the time you should try avoiding doing so and using vectorized operations instead.
• Re documentation I mostly rely on the python documentation + merlin/utop for completion. The generated odoc would only contain function signatures for now - not sure how much help this would be compared to merlin but happy to consider it if it's useful.

[TK997]

Hello and don't mind me asking, but shouldn't this: Tensor.arange1 ~start:(Scalar.int 0) ~end_:(Scalar.int 5) ~options:(Int, Cpu);; generate numbers from 0 to 4?

[LaurentMazare]

Indeed that's what's expected as per the pytorch doc.

You can also use arange to generate float intervals and arange2 gives access to the step argument.

[TK997]

ohh I see, I see, you must add Tensor.print to actually get output :D
However, I am having hard time understanding why we need Tensor. If I am not wrong, Tensor is same as multidimension matrix representation right?

[LaurentMazare]

Indeed, Tensor.t is just a wrapper around PyTorch C++ tensor type (which comes from the ATen library).
(and the Tensor.printf call is probably not needed if you use the latest github version and use dune utop)

[bluddy]

Thanks for the answers! I understand the design decisions now. Thank you for doing this -- it's definitely the closest we can come to having state of the art deep learning in OCaml.

BTW do you know of a good visualization solution? That's the other missing piece of the puzzle, and I don't think there's a clean way to interface with matplotlib.

[LaurentMazare]

No problem, these bindings are quite close to the underlying C++ api, they could certainly use a more 'ocamlish' layer on top.
Concerning visualization I actually wrote some small matplotlib bindings a couple months back, very minimal but they work for my use case (and happy to add support for other plotting types if there is some interest).
For 'interactive' machine learning, I had some good experience using ocaml-jupyter, it works fairly well and you can use ocaml-torch with this. It comes with a small plotting library (archimedes) but I prefer using my matplotlib bindings as I'm more familiar with matplotlib.

[bluddy]

Nice! So pyml works well for this kind of stuff? I'll try to play around with your library, and add stuff as needed.

[LaurentMazare]

Yes pyml should work fine for this. You can even get it to work in a notebook by setting the matplotlib backend to agg, saving the image to say /tmp/a.png and loading it with Jupyter_notebook.display "image/png" "/tmp/a.png" ~base64:true.
I would recommend using the github tip of pyml as it fixes some memory leak and a potential segfault. I'm considering switching this to ocaml-py at some point although I'm not sure about it right now.

[LaurentMazare]

@bluddy do you have any more issue ? I'll close the issue in a couple days if not.

[bluddy]

No I'm good. I'll close it.