Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong searchSize in GridSearchCandidateGenerator #8082

Closed
ch1010 opened this issue Aug 9, 2019 · 1 comment · Fixed by SkymindIO/deeplearning4j#141

Comments

@ch1010
Copy link

commented Aug 9, 2019

Issue Description

The searchSize calculated in GridSearchCandidateGenerator.initialize() is wrong
I have four ParameterSpaces:

ParameterSpace<Double> learningRateHyperparam = new DiscreteParameterSpace<>(0.003, 0.005, 0.01, 0.05);
ParameterSpace<Integer> layerSizeHyperparam1 = new DiscreteParameterSpace<>(32, 64, 96, 128);
ParameterSpace<Integer> layerSizeHyperparam2 = new DiscreteParameterSpace<>(32, 64, 96, 128);
ParameterSpace<Double> dropoutHyperparam = new DiscreteParameterSpace<Double>(0.8, 0.9);

But the searchSize is 16, which is 4 (from layerSizeHyperparam1) x 4 (from layerSizeHyperparam2).
AdamSpace(learningRateHyperparam) and DropoutSpace are not considered.

Version Information

Deeplearning4j version: 1.0.0-beta4

Additional Information

The full structure of MultiLayerSpace:

protected MultiLayerSpace getNetworkSpace() {
        ParameterSpace<Double> learningRateHyperparam = new DiscreteParameterSpace<>(0.003, 0.005, 0.01, 0.05);
        ParameterSpace<Integer> layerSizeHyperparam1 = new DiscreteParameterSpace<>(32, 64, 96, 128);
        ParameterSpace<Integer> layerSizeHyperparam2 = new DiscreteParameterSpace<>(32, 64, 96, 128);
        ParameterSpace<Double> dropoutHyperparam = new DiscreteParameterSpace<Double>(0.8, 0.9);

        return new MultiLayerSpace.Builder()
                .seed(SEED)
                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
                .updater(new AdamSpace(learningRateHyperparam))
                .weightInit(WeightInit.XAVIER)
                .l2(0.0001)
                .addLayer(new DenseLayerSpace.Builder()
                        .nIn(mInputNum)
                        .nOut(layerSizeHyperparam1)
                        .build())
                .addLayer(new BatchNormalizationSpace.Builder()
                        .nOut(layerSizeHyperparam1)
                        .activation(Activation.RELU)
                        .build())
                .addLayer(new DropoutLayerSpace.Builder()
                        .dropOut(dropoutHyperparam)
                        .build())
                .addLayer(new DenseLayerSpace.Builder()
                        .nOut(layerSizeHyperparam2)
                        .build())
                .addLayer(new BatchNormalizationSpace.Builder()
                        .nOut(layerSizeHyperparam2)
                        .activation(Activation.RELU)
                        .build())
                .addLayer(new DropoutLayerSpace.Builder()
                        .dropOut(dropoutHyperparam)
                        .build())
                .addLayer(new OutputLayerSpace.Builder()
                        .nOut(mOutputNum)
                        .activation(Activation.SOFTMAX)
                        .lossFunction(LossFunctions.LossFunction.RECONSTRUCTION_CROSSENTROPY)
                        .build())
                .numEpochs(mHyperEpochCount)
                .build();
}

@AlexDBlack AlexDBlack added this to the 1.0.0-beta5 milestone Aug 14, 2019

@AlexDBlack AlexDBlack self-assigned this Aug 19, 2019

AlexDBlack added a commit to SkymindIO/deeplearning4j that referenced this issue Aug 21, 2019
eclipse#8082 Arbiter - fix GridSearchCandidateGenerator search size i…
…ssue

Signed-off-by: AlexDBlack <blacka101@gmail.com>
AlexDBlack added a commit to SkymindIO/deeplearning4j that referenced this issue Aug 21, 2019
Various fixes (#141)
* eclipse#8121 CnnSentenceDataSetIterator fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8120 CnnSentenceDataSetIterator.loadSingleSentence no words UX/exception improvement

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8122 AggregatingSentenceIterator builder - addSentencePreProcessor -> sentencePreProcessor

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8082 Arbiter - fix GridSearchCandidateGenerator search size issue

Signed-off-by: AlexDBlack <blacka101@gmail.com>
@AlexDBlack

This comment has been minimized.

Copy link
Contributor

commented Aug 21, 2019

Thanks for reporting - fixed here, will be merged to eclipse/deeplearning4j master soon.
SkymindIO#141

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.