Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash when I try to get sample the dataset #5477

Closed
ragnarok22 opened this issue Jun 5, 2018 · 10 comments

Comments

@ragnarok22
Copy link

commented Jun 5, 2018

Hi guys! I have an issue. after load an image dataset i'm trying to get a sample using this:
data.sample(10, true)
data is a DataSet. It throws me the next error:
java.lang.IllegalStateException: Rank is 4 columns() call is not valid
How can I fix it? I use 0.9.1 version

@ragnarok22

This comment has been minimized.

Copy link
Author

commented Jun 5, 2018

As I can see in the sample() method implementation, the first line called getfeatures().columns(). It seems that the columns() method is not supported for image data set since it is of 4 dimensions

@ragnarok22

This comment has been minimized.

Copy link
Author

commented Jun 5, 2018

I thinks that the columns in a image dataset is width * height * channels

@raver119

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2018

It's just some legacy method, that was written with regular fully-connected networks in mind.

So it just should be updated for features/labels >2D

@ragnarok22

This comment has been minimized.

Copy link
Author

commented Jun 6, 2018

how can i do that?

@ragnarok22

This comment has been minimized.

Copy link
Author

commented Jun 6, 2018

as far I can see, this method never work with replacement because Set<Integer> added are queried but never updated

@AlexDBlack

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2018

We need to update the code to do that.
In the mean time, you can use the DataSet.asList() + randomly sample using standard java RNG/list ops + DataSet.merge()

@ragnarok22

This comment has been minimized.

Copy link
Author

commented Jun 7, 2018

Thanks. I will try to do that way. Can you label this to bug and assign it?

@AlexDBlack AlexDBlack added the Bug label Jun 7, 2018

@AlexDBlack

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2018

@ragnarok22 I don't think people without write access will be able to assign, but we can. Did you want this assigned to you? (i.e., are you planning on submitting a fix?)
Edit: if not, we'll assign it when someone starts working on it. Possibly me, we'll see :)

@ragnarok22

This comment has been minimized.

Copy link
Author

commented Jun 7, 2018

I would gladly do it, but at this moment I am doing my undergraduate thesis and I do not have much time. Very soon I will be able to help you :)

@AlexDBlack AlexDBlack added this to the DL4J/Arbiter/DataVec Planning Sprint (01 June) milestone Jun 7, 2018

@AlexDBlack AlexDBlack self-assigned this Jun 7, 2018

AlexDBlack added a commit that referenced this issue Jun 12, 2018
AlexDBlack added a commit that referenced this issue Jun 14, 2018
@lock

This comment has been minimized.

Copy link

commented Sep 21, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Sep 21, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.