Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transferlearning#nOutReplace reverts changed number of inputs to next layer if that layer is also changed #6343

Closed
DrChainsaw opened this issue Sep 2, 2018 · 1 comment

Comments

@DrChainsaw
Copy link

commented Sep 2, 2018

The testcase below fails on windows 10 using beta 2 (both CPU and CUDA 9.2).

The testcase attempts to change nOut of two layers which are subsequent to each other, which results in nIn to the second being overwritten to the original value. Inspecting the code in TransferLearning#nOutReplace confirms this is the case as the config to modify is taken from the original graph and the modified config is put directly into the editedConfigBuilder, overwriting any prior changes.

Taking the config to modify from the editedConfigBuilder or from editedVertices if present could be a way forward.

   @Test
   public void testTransferLearningSubsequent() {
       final String firstConv = "firstConv";
       final String secondConv = "secondConv";
       final INDArray input = Nd4j.create(6,6,6,6);
       final ComputationGraph graph = new ComputationGraph(new NeuralNetConfiguration.Builder()
               .weightInit(new ConstantDistribution(666))
               .graphBuilder()
               .addInputs(inputName)
               .setOutputs(outputName)
               .setInputTypes(InputType.inferInputTypes(input))
               .addLayer(firstConv, new Convolution2D.Builder(3, 3)
                       .nOut(10)
                       .build(), inputName)
               .addLayer(secondConv, new Convolution2D.Builder(1, 1)
                       .nOut(5)
                       .build(), firstConv)
               .addLayer(outputName, new OutputLayer.Builder()
                       .nOut(2)
                       .build(), secondConv)
               .build());
       graph.init();

       final ComputationGraph newGraph = new TransferLearning
               .GraphBuilder(graph)
               .nOutReplace(firstConv, 7, new ConstantDistribution(333))
               .nOutReplace(secondConv, 3, new ConstantDistribution(111))
               .build();
       newGraph.init();

       // newGraph now has input size of 10 for secondConv!
       assertEquals("Incorrect nrof inputs!", 7, ((FeedForwardLayer)newGraph.getLayer(secondConv).conf().getLayer()).getNIn());

       // Crash!
       newGraph.outputSingle(input);
   }

@AlexDBlack AlexDBlack self-assigned this Sep 3, 2018

AlexDBlack added a commit that referenced this issue Sep 3, 2018
AlexDBlack added a commit that referenced this issue Sep 4, 2018
ND4J indexing fixes + DL4J fix (#6353)
* #6327 INDArray.put with SpecifiedIndex

* #6341 - SpecifiedIndex with single value no longer collapses dimensions

* Another indexing fix

* #6343 TransferLearning nOutReplace fix
@lock

This comment has been minimized.

Copy link

commented Oct 4, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Oct 4, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants
You can’t perform that action at this time.