Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mlpML tuneGrid checking implementation flawed #829

Closed
Twitchys opened this issue Feb 7, 2018 · 4 comments
Closed

mlpML tuneGrid checking implementation flawed #829

Twitchys opened this issue Feb 7, 2018 · 4 comments

Comments

@Twitchys
Copy link

@Twitchys Twitchys commented Feb 7, 2018

The mlpML model treats a nn size definition of c(5,0,7) as the same as (5), ignoring anything following an empty layer, but not producing an error. The mlpML caret implementation has checking to avoid this in line 28 of mlpML.R, however, the logic is incorrectly implemented.
if(param$layer2 == 0 & param$layer2 > 0) stop("the second layer must have at least one hidden unit if a third layer is specified")
should be
if(param$layer2 == 0 & param$layer3 > 0) stop("the second layer must have at least one hidden unit if a third layer is specified")

Minimal, runnable code:

library(caret)
set.seed(1)
dat <- twoClassSim(100)
X <- dat[,1:5]
y <- dat[["Class"]]

mlpMLGrid <- data.frame(layer1=5,
                        layer2=0,
                        layer3=7)

model_class <- train(X, y, method='mlpML', tuneGrid=mlpMLGrid)
print(model_class$finalModel$archParams)

Session Info:

sessionInfo()

Thanks!

@topepo
Copy link
Owner

@topepo topepo commented Feb 7, 2018

There is a bug in the code but not for what you show. It is supposed to stop execution if the second layer has zero units since that is nonsensical (to me, at least). The intention is to convert your specification above to be c(5, 7) instead of c(5, 0, 7). I'm open to suggestions if you disagree.

@Twitchys
Copy link
Author

@Twitchys Twitchys commented Feb 7, 2018

It looks like mxnet treats c(5,0,7) as c(5,7) so it would be consistent with that. On the other hand, the neuralnet model seems to treat c(5,0,7) as c(5), so there's not existing consensus among models. I'm fairly new to the field so I don't have an intuition for how the community would want to interpret it, so I'd defer to your opinion.

@topepo topepo added the reproducible label Feb 7, 2018
@topepo topepo added the next release label Feb 27, 2018
topepo added a commit that referenced this issue Mar 6, 2018
@topepo
Copy link
Owner

@topepo topepo commented Mar 6, 2018

How does that look?

@Twitchys
Copy link
Author

@Twitchys Twitchys commented Mar 7, 2018

That looks perfect! Clears up any ambiguity with a good warning message.

@topepo topepo closed this Mar 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.