Pretty Print Tensors #257

agelas · 2023-03-30T07:58:43Z

@nathanielsimard I'm new to Rust so I figured I'd give #138 a shot, Let me know if this is somewhere in the ballpark of what you were thinking. I only implemented a pretty print for 2D Tensor structs composed of Ints (or at least that's what I think I did 🙃), and I can generalize to other types later.

burn-tensor/src/tensor/api/base.rs

antimora · 2023-03-30T16:37:09Z

burn-tensor/src/tensor/api/base.rs

+    fn to_nested_vec(&self) -> Vec<Vec<B::IntElem>> {
+        let data = self.to_data();
+        let mut result = vec![vec![B::IntElem::default(); self.dims()[1]]; self.dims()[0]];
+        for (i, val) in data.value.iter().enumerate() {
+            let row = i / self.dims()[1];
+            let col = i % self.dims()[1];
+            result[row][col] = *val;
+        }
+        result


This could be made as a separate standalone function that can take the multi dimensional tensor and return formatted string string. This way you don't need to worry about nested vectors.

The challenging part is to convert the flat data into nested array. But I am sure it's doing. Here is an example output how numpy does:

>>> np.ones([15, 15,15, 15]) array([[[[1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], ..., [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.]], [[1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], ..., [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.]], [[1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], ..., [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.]], ..., [[1., 1., 1., ..., 1., 1., 1.], [1., 1., 1., ..., 1., 1., 1.],

My current strategy is to try and recursively format the tensor as a 2D vector for each dimension, indenting based on the dimensionality level. It's still a work in progress because I need to figure out how to properly slice up tensors.

I think a recursive strategy is the only way to generalize it to any dimension, or at least the only strategy that I can think of 😅.

@agelas @nathanielsimard

I came across this project serde-dim that looks interesting. Maybe there is a something you can learn from it: https://github.com/RReverser/serde-ndim/blob/main/src/ser.rs . What this project is trying to accomplish is basically convert one dimensional data array into multi dimensional array. Pretty what you're trying to accomplish.

@antimora Looks interesting, I'll give it a look. But your description seems almost the opposite of what this PR is for, ie we're going from multi-dimensional array -> "1-dimensional" string representation and not the other way around.

@antimora Looks interesting, I'll give it a look. But your description seems almost the opposite of what this PR is for, ie we're going from multi-dimensional array -> "1-dimensional" string representation and not the other way around.

The tensor data is stored as one-dimensional vector and shape contains the dimensions so you know how to decompose into rows and columns. The link I shared basically is trying to take the serialized data (1d) and deserialize into n-dimensional array by recursion. By coincidence I came across this code and I thought there might be something to learn. I haven't dived deep to see if this direct helps you but in principle by its description it should. I hope it does not confuse you =)

…imensionality as lists of 1D vectors

burn-tensor/src/tensor/api/base.rs

nathanielsimard

This is starting to look good! I think we should make the implementation generic over the kind, and with the other minor requested changes we could merge the PR pretty soon.

nathanielsimard · 2023-04-03T22:47:00Z

burn-tensor/src/tensor/api/base.rs

+    pub fn size(&self) -> usize {
+        self.dims().iter().fold(1, |acc, &dim| acc * dim)
+    }


There is a similar method num_elements in the Shape struct, I would use it instead here, but I like the fold implementation, so we may update the num_elements function implementation to use fold instead of loops. size is a bit unclear as a function name, since size often return the shape in PyTorch, so we may update the name to num_elements.

Actually I didn't mean to leave this in- I needed something like this for an earlier implementation I was trying out. We can remove it, rename it, or do whatever honestly 🤣

burn-tensor/src/tensor/api/base.rs

burn-tensor/src/tests/stats/basic.rs

burn-tensor/src/tensor/api/base.rs

agelas · 2023-04-04T05:11:02Z

burn-tensor/src/tensor/api/base.rs

@@ -212,6 +307,19 @@ pub trait BasicOps<B: Backend>: TensorKind<B> {
        rhs: Self::Primitive<D>,
    ) -> Tensor<B, D, Bool>;
    fn equal_elem<const D: usize>(lhs: Self::Primitive<D>, rhs: Self::Elem) -> Tensor<B, D, Bool>;
+    fn elem_type() -> &'static str {
+        if TypeId::of::<Self::Elem>() == TypeId::of::<f32>() {


@nathanielsimard Also here I assume I'm missing a bunch of other possibilities like f64 for float, or i8, i16, i128, etc. for ints. What else should I add?

You can use directly core::any::type_name::<Seld::Elem>() and all types will be supported :), I would also rename the function to elem_type_name to be more clear.

Do you want the more specific type that comes from core::any::type_name::<Self::Elem>(), or should I group the possible returns and return int/float?

The more specific type is needed, but we could also add the tensor kind:

data: [...], kind: Float, // We could add a function `name` in the trait TensorKind. elem: f32

Sounds good, just threw kind in as well.

…c type names

…d fixed up UT

agelas · 2023-04-06T01:03:56Z

@nathanielsimard I think this is pretty much done unless you can think of anything else. I just merged main so the branch is all up to date as well.

nathanielsimard · 2023-04-06T14:09:30Z

@agelas I think the only missing part is fixing the CI (mostly no_std support by importing stuff from alloc) and remove the size function from the tensor API.

…ttyPrint

agelas · 2023-04-06T19:46:54Z

@nathanielsimard Ok should be good now, although locally I can't seem to build because the compiler is complaining about the "use of unstable library feature 'unzip_option': recently added" in burn-core/src/optim/simple/adaptor.rs:86:63. Not sure if that will trip up the CI as well.

nathanielsimard · 2023-04-06T19:56:55Z

@nathanielsimard Ok should be good now, although locally I can't seem to build because the compiler is complaining about the "use of unstable library feature 'unzip_option': recently added" in burn-core/src/optim/simple/adaptor.rs:86:63. Not sure if that will trip up the CI as well.

@agelas I'm curious, what is your local Rust version? Maybe I used a newly stabilized function, and I should update the minimum required Rust version in the readme. The CI is always using the latest stable Rust version.

agelas · 2023-04-06T20:01:14Z

@nathanielsimard Oh I'm on 1.65.0, that might explain it.

nathanielsimard · 2023-04-06T20:49:18Z

@agelas You can run clippy and cargo fmt with the lastest Rust to fix most problems.

cargo clippy --fix
cargo fmt --all

agelas · 2023-04-06T21:56:28Z

@nathanielsimard clippy and fmt didn't seem to change anything. I added a few more imports from alloc though hopefully that should fix it.

agelas · 2023-04-06T22:12:05Z

@nathanielsimard Ok we're getting a little bit closer 😆. The test-burn-ndarray one is strange. I've noticed that sometimes the backend comes back as ndarray, sometimes as tch, do you know what might be up with that?

nathanielsimard · 2023-04-06T22:16:43Z

@nathanielsimard Ok we're getting a little bit closer laughing. The test-burn-ndarray one is strange. I've noticed that sometimes the backend comes back as ndarray, sometimes as tch, do you know what might be up with that?

I think, the tests in ndarray will have ndarray as backend and the ones in tch will have tch. But backends can be used as dev dependencies just to run the tests, so it might change. I would sugest to update the test to use the TestBackend name directly.

agelas · 2023-04-06T22:30:17Z

I think, the tests in ndarray will have ndarray as backend and the ones in tch will have tch. But backends can be used as dev dependencies just to run the tests, so it might change. I would sugest to update the test to use the TestBackend name directly.

Sorry, not quite sure what you mean by using the TestBackend name directly. Do you mean explicitly doing something like let TestBackend = burn_ndarray::NdArrayBackend<f32>?

nathanielsimard · 2023-04-06T22:51:29Z

Sorry, not quite sure what you mean by using the TestBackend name directly. Do you mean explicitly doing something like let TestBackend = burn_ndarray::NdArrayBackend<f32>?

No I mean in the test, instead of hardcoding the name of the backend in the expected string, maybe use a format!("... backend: {:?}...", TestBackend::name())

agelas · 2023-04-06T23:19:01Z

@nathanielsimard I'm just gonna remove the doctest lol.

agelas · 2023-04-06T23:33:16Z

@nathanielsimard alright this time I'm 99% sure it should pass

agelas added 5 commits March 28, 2023 22:53

Start implementation of B::IntElem pretty print

0d1eab3

Specify this is for Ints right now

5abb5fc

Work on test

e18b845

Attempt 2 at figuring out imports

11b278a

Fix test

e96b25c

agelas marked this pull request as draft March 30, 2023 07:59

antimora requested changes Mar 30, 2023

View reviewed changes

agelas added 4 commits March 31, 2023 01:09

Should be another newline in 2d and possible test for 3d tensor

8129794

Try to generalize to arbitrary dimensions, still WIP

e92a7ce

Use recursive strategy to display elements of tensor with arbitrary d…

7bb4748

…imensionality as lists of 1D vectors

Added 3 and 4D tensor tests

6501511

agelas commented Apr 2, 2023

View reviewed changes

burn-tensor/src/tensor/api/base.rs Outdated Show resolved Hide resolved

agelas added 3 commits April 2, 2023 18:20

Format

0adb802

Impl for float and add float tensor to test

dff174a

Impl for bool tensor and add to test to cover bool tensors

a039595

agelas marked this pull request as ready for review April 3, 2023 05:59

nathanielsimard requested changes Apr 3, 2023

View reviewed changes

agelas added 2 commits April 3, 2023 21:43

Make std::fmt::Display and display_recursive generic over K: BasicOps

f0eba0a

One assert per test

c59cfcf

agelas commented Apr 4, 2023

View reviewed changes

burn-tensor/src/tensor/api/base.rs Show resolved Hide resolved

agelas commented Apr 4, 2023

View reviewed changes

agelas and others added 3 commits April 4, 2023 23:14

Renamed elem_type to elem_type_name, fixed ut to reflect more specifi…

92e70c3

…c type names

Added name() function to TensorKind, updated Display to show kind, an…

d23d99a

…d fixed up UT

Merge branch 'burn-rs:main' into PrettyPrint

48c92ef

agelas added 4 commits April 6, 2023 12:37

Use core instead of std

6fdf762

Backend is tch instead of ndarray now

ab71a81

Merge branch 'PrettyPrint' of https://github.com/agelas/burn into Pre…

21d2f29

…ttyPrint

Format

5f76392

Added format and vec imports

9f80c29

More imorts for CI

4326d10

This time for sure

b6da8f2

agelas added 2 commits April 6, 2023 16:19

Remove doctest

9595dd4

Add alloc::format to tests mod

94a8181

nathanielsimard merged commit d8f64ce into tracel-ai:main Apr 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pretty Print Tensors #257

Pretty Print Tensors #257

agelas commented Mar 30, 2023

antimora Mar 30, 2023

agelas Mar 31, 2023

nathanielsimard Mar 31, 2023

antimora Apr 3, 2023

agelas Apr 3, 2023

antimora Apr 3, 2023

nathanielsimard left a comment

nathanielsimard Apr 3, 2023

agelas Apr 4, 2023

agelas Apr 4, 2023

nathanielsimard Apr 4, 2023 •

edited

Loading

agelas Apr 5, 2023 •

edited

Loading

nathanielsimard Apr 5, 2023

agelas Apr 5, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023 •

edited

Loading

agelas commented Apr 6, 2023

agelas commented Apr 6, 2023

Pretty Print Tensors #257

Pretty Print Tensors #257

Conversation

agelas commented Mar 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nathanielsimard left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nathanielsimard Apr 4, 2023 • edited Loading

Choose a reason for hiding this comment

agelas Apr 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard commented Apr 6, 2023 • edited Loading

agelas commented Apr 6, 2023

agelas commented Apr 6, 2023

nathanielsimard Apr 4, 2023 •

edited

Loading

agelas Apr 5, 2023 •

edited

Loading

nathanielsimard commented Apr 6, 2023 •

edited

Loading