New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INDArray#put(INDArrayIndex[], INDArray) does not work for more than one dimension with SpecifiedIndex #6327

Closed
DrChainsaw opened this Issue Sep 1, 2018 · 5 comments

Comments

Projects
None yet
2 participants
@DrChainsaw
Copy link
Contributor

DrChainsaw commented Sep 1, 2018

Hi,

The following testcase fails on windows 10 using beta2 (both native and GPU):

    @Test
    public void testPutINDArrayIndex() {

        // Works for one dimension
//        final long[] shapeSource = {3};
//        final long[] shapeTarget = {4};
//        final long[] diffShape =   {1};

        // Does not work for two dimensions
        final long[] shapeSource = {3, 4};
        final long[] shapeTarget = {4, 4};
        final long[] diffShape = {1, 4};

        // Does not work for three dimensions either...
//        final long[] shapeSource = {3,4,5};
//        final long[] shapeTarget = {4,4,5};
//        final long[] diffShape =   {1,4,5};

        // ...and neither for four...
//        final long[] shapeSource = {3,4,5,6};
//        final long[] shapeTarget = {4,4,5,6};
//        final long[] diffShape =   {1,4,5,6};

        final INDArray source = Nd4j.ones(shapeSource);
        final INDArray target = Nd4j.zeros(shapeTarget);

        final INDArrayIndex[] sourceIndexes = new INDArrayIndex[shapeSource.length];
        Arrays.fill(sourceIndexes, NDArrayIndex.all());

        final INDArrayIndex[] targetIndexes = new INDArrayIndex[shapeTarget.length];
        Arrays.fill(targetIndexes, NDArrayIndex.all());
        // Doesn't work
        targetIndexes[0] = new SpecifiedIndex((IntStream.range(0, (int)shapeSource[0]).toArray()));

        // Works
        //targetIndexes[0] = NDArrayIndex.interval(0, shapeSource[0]);

        target.put(targetIndexes, source);
        final INDArray expected = Nd4j.vstack(Nd4j.ones(shapeSource), Nd4j.zeros(diffShape));
        assertEquals("Expected array to be set!", expected, target);
    }

@AlexDBlack AlexDBlack self-assigned this Sep 3, 2018

@AlexDBlack

This comment has been minimized.

Copy link
Member

AlexDBlack commented Sep 3, 2018

So, the explanaition is pretty straightforward for this bug: put is basically get(indices).assign(array).
However, when using SpecifiedIndex, getting a view is not possible, so we've updating a copy with values that has been pulled from the array, not the original array itself...

@DrChainsaw

This comment has been minimized.

Copy link
Contributor

DrChainsaw commented Sep 3, 2018

Thanks for the explanation, makes sense.

I guess it is possible to device some kind of tensoralongdimension loop as a workaround, or are there better ways? I think you mentioned Nd4j.pullRows in chat the other day.

@DrChainsaw DrChainsaw closed this Sep 3, 2018

@DrChainsaw

This comment has been minimized.

Copy link
Contributor

DrChainsaw commented Sep 3, 2018

Finger slipped on phone. Sorry...

@DrChainsaw DrChainsaw reopened this Sep 3, 2018

@AlexDBlack

This comment has been minimized.

Copy link
Member

AlexDBlack commented Sep 3, 2018

Yeah, I've implemented (but haven't yet pushed up) something exactly like that.
i.e., iterate over the specified indices, but (essentially) keep the other indices the same.
So we'll copy rank R=origRank-numSpecifiedAxis sub-arrays at each step of the loop.

AlexDBlack added a commit that referenced this issue Sep 4, 2018

ND4J indexing fixes + DL4J fix (#6353)
* #6327 INDArray.put with SpecifiedIndex

* #6341 - SpecifiedIndex with single value no longer collapses dimensions

* Another indexing fix

* #6343 TransferLearning nOutReplace fix
@lock

This comment has been minimized.

Copy link

lock bot commented Oct 4, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Oct 4, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.