Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the tune_new_entries parameter really do what the documentation says? #40

Open
mikulskibartosz opened this issue Jul 30, 2019 · 0 comments

Comments

@mikulskibartosz
Copy link

commented Jul 30, 2019

Summary

Hi,
I have been training HyperResNet using the Hyperband tuner, and I have noticed that when I set the tune_new_entries parameter to True, the parameters which I did not specify manually are not even listed in the tuner's output. After reading the documentation, I was the under impression that the tuner is supposed to add those parameters to the search space and tune them.

On the other hand, when I set the tune_new_entries to False, the tuner lists all parameters of the HyperResNet and chooses a different value every time I run it.

Steps to reproduce the problem

from tensorflow import keras

fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

train_images = train_images / 255.0
test_images = test_images / 255.0

train_images = train_images.reshape(len(train_images), 28, 28, 1)
test_images = test_images.reshape(len(test_images), 28, 28, 1)

from keras.utils import to_categorical
train_labels_binary = to_categorical(train_labels)

from kerastuner.applications import HyperResNet
from kerastuner.tuners import Hyperband

hypermodel = HyperResNet(input_shape=(28, 28, 1), classes=10)

from kerastuner import HyperParameters
hp = HyperParameters()
hp.Choice('learning_rate', values=[1e-3, 1e-4])
hp.Fixed('optimizer', value='adam')

tuner = Hyperband(
    hypermodel,
    objective='val_accuracy',
    hyperparameters=hp,
    tune_new_entries=True,
    max_trials=20,
    directory='FashionMnistResNet',
    project_name='FashionMNIST')

tuner.search(train_images, train_labels_binary, validation_split=0.1)

Expected results

The list of tuned arguments printed to the output should also contain values not explicitly specified as the instance of HyperParameters, because according to the documentation tune_new_entries=False prevents unlisted parameters from being tuned. I set it to True, so I expect the unlisted parameters to be tuned. That does not seem to happen.

Actual results

This is printed:

Hp values:
|-learning_rate: 0.0001
|-optimizer: adam
|-tuner/epochs: 3

When I set tune_new_entries=False, it prints this:

|-learning_rate: 0.0001
|-optimizer: adam
|-pooling: max
|-tuner/epochs: 3
|-v2/conv3_depth: 4
|-v2/conv4_depth: 36
|-version: v2

I guess it should be the other way around.

Versions

Keras: 2.2.4-tf
Tensorflow: 2.0.0-beta1
Keras tuner: installed when cfc6e20956cb8554ee29ef2a1ba4635da7d0228b commit was the most recent one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.