TypeError when sorting candidates during Hyperband search

[floscha]

The following simple toy example fails with the error message shown below. When using the RandomSearch tuner instead, everything works as expected.

def build_model(hp):
    inputs = layers.Input(shape=(5, ))
    x = layers.Dense(units=hp.Range('units', min_value=32, max_value=512, step=32),
                         activation='relu')(inputs)
    predictions = layers.Dense(1)(x)
    model = keras.models.Model(inputs=inputs, outputs=predictions)
    model.compile(optimizer='adam', loss='mean_squared_error')
    return model

tuner = kerastuner.tuners.Hyperband(
    build_model,
    objective='val_loss',
    max_trials=2,
    executions_per_trial=1
)

tuner.search(np.eye(5), np.ones((5, 1)),
             validation_data=(np.eye(5), np.ones((5, 1))),
             epochs=2)

1/2 trials left
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-c7537b51ff32> in <module>
     19 tuner.search(np.eye(5), np.ones((5, 1)),
     20              validation_data=(np.eye(5), np.ones((5, 1))),
---> 21              epochs=2)

/opt/conda/lib/python3.6/site-packages/kerastuner/engine/tuner.py in search(self, *fit_args, **fit_kwargs)
    207             # Obtain unique trial ID to communicate with the oracle.
    208             trial_id = tuner_utils.generate_trial_id()
--> 209             hp = self._call_oracle(trial_id)
    210             if hp is None:
    211                 # Oracle triggered exit

/opt/conda/lib/python3.6/site-packages/kerastuner/engine/tuner.py in _call_oracle(self, trial_id)
    525         # Obtain hp value suggestions from the oracle.
    526         while 1:
--> 527             oracle_answer = self.oracle.populate_space(trial_id, hp.space)
    528             if oracle_answer['status'] == 'RUN':
    529                 hp.values = oracle_answer['values']

/opt/conda/lib/python3.6/site-packages/kerastuner/tuners/hyperband.py in populate_space(self, trial_id, space)
     86         if self._bracket_index + 1 < self._num_brackets:
     87             self._bracket_index += 1
---> 88             self._select_candidates()
     89         # If the current band ends
     90         else:

/opt/conda/lib/python3.6/site-packages/kerastuner/tuners/hyperband.py in _select_candidates(self)
    135     def _select_candidates(self):
    136         sorted_candidates = sorted(list(range(len(self._candidates))),
--> 137                                    key=lambda i: self._candidate_score[i])
    138         num_selected_candidates = self._model_sequence[self._bracket_index]
    139         for index in sorted_candidates[:num_selected_candidates]:

TypeError: '<' not supported between instances of 'NoneType' and 'float'

The code was run with Python 3.6.6 and the following relevant libraries:

tensorflow                         2.0.0b1      
Keras-Tuner                        0.9.0.1562790722 
numpy                              1.16.4 

[SivamPillai]

Also facing the same issue using a different dataset. @floscha Were you able to find a solution to this problem as yet? Random Search works fine also for me.

[floscha]

Hey @SivamPillai. Unfortunately, I couldn't find the time yet to look deep enough into the code to see where the error lies. But since I'm apparently not the only one with this problem, I'll see what I can do within the next couple of days, unless of course, @jhfjhfj1 does so as he's certainly more familiar with the codebase.

[jaredcolerosenberg]

I don't know the exact reason why None is appended to self._candidate_score (I tried to trace through the code but I am still learning exactly how Hyperband works), but that is the root of the problem. If you instead append a large number (largest possible score b/c they are sorted later on), then it seems to work.

  def _generate_candidates(self):
        self._candidates = []
        self._candidate_score = []
        num_models = self._model_sequence[0]

        for index in range(num_models):
            instance = self._new_trial()
            if instance is not None:
                self._candidates.append(instance)
                self._candidate_score.append(None)

        for index, instance in enumerate(self._candidates):
            self._queue.put(index)

[mikulskibartosz]

How I found a possible fix

I got it working, by using some monkey-patching in the runtime. I had to redefine the _select_candidates function inside a HyperbandOracle instance used by the tuner.

from functools import total_ordering
import types

@total_ordering
class MaxType(object):
    def __le__(self, other):
        return False

    def __eq__(self, other):
        return (self is other)

Max = MaxType()

def _select_candidates(self):
        sorted_candidates = sorted(list(range(len(self._candidates))),
                                   key=lambda i: Max if self._candidate_score[i] is None else self._candidate_score[i])
        num_selected_candidates = self._model_sequence[self._bracket_index]
        for index in sorted_candidates[:num_selected_candidates]:
            self._queue.put(index)

tuner.oracle._select_candidates = types.MethodType(_select_candidates, tuner.oracle)

The suggested change

Right now, I think we can get it fixed by:

1. Defining the MaxType used for comparison in case of a None value
2. Replacing the key lambda in _select_candidates function with: lambda i: Max if self._candidate_score[i] is None else self._candidate_score[i]

What do you think?

[samuelBedard]

facing the same issue

[SivamPillai]

How I found a possible fix

I got it working, by using some monkey-patching in the runtime. I had to redefine the _select_candidates function inside a HyperbandOracle instance used by the tuner.

from functools import total_ordering
import types

@total_ordering
class MaxType(object):
    def __le__(self, other):
        return False

    def __eq__(self, other):
        return (self is other)

Max = MaxType()

def _select_candidates(self):
        sorted_candidates = sorted(list(range(len(self._candidates))),
                                   key=lambda i: Max if self._candidate_score[i] is None else self._candidate_score[i])
        num_selected_candidates = self._model_sequence[self._bracket_index]
        for index in sorted_candidates[:num_selected_candidates]:
            self._queue.put(index)

tuner.oracle._select_candidates = types.MethodType(_select_candidates, tuner.oracle)

The suggested change

Right now, I think we can get it fixed by:

1. Defining the MaxType used for comparison in case of a None value
2. Replacing the key lambda in _select_candidates function with: lambda i: Max if self._candidate_score[i] is None else self._candidate_score[i]

What do you think?

This fixed it for me. Thanks.

[rcmagic1]

I have this issue as well and this fix appears to work for me.
Has an official fix been merged into master?

[franchesoni]

I have this issue as well and this fix appears to work for me.
Has an official fix been merged into master?

Same

[floscha]

While the "fix" makes the code run, can anyone confirm that it actually works? When debugging the self._candidate_score list, I noticed that all trials except one are None which is most certainly not the desired behavior. So, even with the code running after the fix, it still appears that one only gets the result of one random trial while ignoring all other trials.

[franchesoni]

I'm not sure if I get that. Code runs and 'best model' metrics (val loss, etc.) seem correct.
Is that _candidate_score list part of Hyperband algorithm or is it the result of the evaluation of a trial (independently of the algorithm)?

I have never used Hyperband before. My "bandit problem" is defined in a very big space, so I could not tell the difference between random search and this.

For the record, I saw that one specific model was evaluated twice (and I'm not talking about variance-reducing repetition).

[omalleyt12]

This should be fixed by: #115

Note that there was a rewrite to the Hyberband implementation, in order to:

1. Fix some issues such as this.
2. Be more faithful to the paper. The implementation now corresponds to Algorithm 1 in the paper
3. Be more efficient when running distributed tuning.

API Changes:

• max_trials replaced with hyperband_iterations (See docstring for description)
• max_epochs is mandatory (The previous default was too low for many models)
• min_epochs is removed (This arg was confusing and really corresponded to an internal implementation detail of Hyperband)

Closing this issue since the error should now be outdated, but if anyone runs into errors with the new implementation please open a new issue.

Thanks!