Getting some kind of AttributeError

[Liquidten]

Traceback (most recent call last):
File "talosHyper.py", line 200, in
experiment_no='1')
File "/usr/local/lib/python3.6/dist-packages/talos/scan/Scan.py", line 166, in init
self._null = self.runtime()
File "/usr/local/lib/python3.6/dist-packages/talos/scan/Scan.py", line 170, in runtime
self = scan_prepare(self)
File "/usr/local/lib/python3.6/dist-packages/talos/scan/scan_prepare.py", line 62, in scan_prepare
self.last_neuron = last_neuron(self)
File "/usr/local/lib/python3.6/dist-packages/talos/utils/last_neuron.py", line 3, in last_neuron
labels = list(set(self.y.flatten('F')))
File "/usr/local/lib/python3.6/dist-packages/pandas/core/generic.py", line 4376, in getattr
return object.getattribute(self, name)
AttributeError: 'Series' object has no attribute 'flatten'

[mikkokotila]

How does your params dictionary look like?

[Liquidten]

p = {'lr': (0.01, 0.1, 1),
     'first_neuron': [4, 8, 16, 32, 64, 128],
     'hidden_layers': [8, 32, 64],
     'batch_size': [5, 10, 15],
     'epochs': [10,15],
     'dropout': (0, 0.50, 1),
     'weight_regulizer': [None],
     'optimizer': [Adam],
     'losses': [binary_crossentropy],
     'activation': [relu, elu],
     'last_activation': [sigmoid]}

This is how I have my parameter setup.

[mikkokotila]

There seems to be something going on with the input data. Can you get the shape of your input data and give a few rows of 'y' as an example.

[Liquidten]

So I am doing text classification using embedding layers.
input data/X_data shape = (280, 106656)
Y_labels shape= (280,)
X_data = [[ 8 138 531 ... 0 0 0]
[ 80 139 289 ... 0 0 0]
[671 162 310 ... 0 0 0]
...
[ 51 29 274 ... 0 0 0]
[168 52 168 ... 0 0 0]
[713 310 808 ... 0 0 0]]
y_label = 0 0
0 0
0 0
0 1
0 0

Data are all numpy.ndarray format.

[Liquidten]

Any feedback?

[mikkokotila]

This can't be the actual y format:

y_label = 0 0
0 0
0 0
0 1
0 0

Can you share the actual python format for the 'y' you input to the model.

[Liquidten]

print(np.array(y_label))
[0 0 0 1 0 1 1 0 0 1 0 1 0 0 1 1 0 1 0 1 0 0 1 1 1 0 0 0 0 0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 1 0 1 1 1 1 0 1 0 0 1 1 1 0 1 0 1 1 0 1 0 1 0 1 1 1 1 1 1 1
1 1 0 0 0 1 1 1 1 0 1 0 1 0 1 1 0 1 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 0 1
1 0 0 0 1 0 0 0 1 1 1 1 0 1 0 0 1 0 1 0 0 0 0 1 1 1 1 0 1 1 1 1 0 0 0 0 0
1 0 0 1 1 0 0 0 0 0 1 1 1 1 0 1 0 1 1 0 1 1 1 1 0 0 0 1 1 1 0 0 1 1 0 1 0
1 0 0 1 0 0 0 1 0 1 1 1 0 0 0 0 1 0 1 0 1 1 0 0 1 0 1 1 1 1 0 0 1 1 0 0 1
0 0 1 1 0 0 1 1 0 0 1 0 0 1 0 0 1 1 0 0 0 1 1 1 1 0 1 1 1 0 1 1 0 1 0 0 0
1 1 1 0 1 1 0 0 0 1 1 1 1 0 1 1 1 1 0 0 1]

The format is a np.array.

[mikkokotila]

Maybe related with the data being passed to the input model. Can you share your input model and also the Scan() command.

[Liquidten]

def create_model(X_train, X_dev, X_test, y_label, y_dev, Test_label, vocab_size,MAXLEN, params):

# next we can build the model exactly like we would normally do it
e = Embedding(vocab_size,150,input_length=MAXLEN)
 
 
model = Sequential()

model.add(e)
model.add(GlobalMaxPooling1D())

model.add(Dense(16, activation=params['activation'],
                kernel_initializer='normal'))

model.add(Dropout(params['dropout']))

# if we want to also test for number of layers and shapes, that's possible
hidden_layers(model, params, 1)

# then we finish again with completely standard Keras way
model.add(Dense(1, activation=params['last_activation'],
                kernel_initializer='normal'))

model.compile(loss=params['losses'],
              # here we add a regulizer normalization function from Talos
              optimizer=params['optimizer'](lr=lr_normalizer(params['lr'],params['optimizer'])),
              metrics=['acc', fmeasure_acc])
model.summary()
history = model.fit(np.array(X_train), np.array(y_label), 
                    validation_data = (np.array(X_dev), np.array(y_dev)),
                    batch_size=params['batch_size'],
                    epochs=params['epochs'],
                    verbose=0)

# finally we have to make sure that history object and model are returned
return history, model

t = ta.Scan(x=(np.array(X_train)),
            y=(np.array(y_label)),
            model=create_model,
            grid_downsample=0.01, 
            params=p,
            dataset_name='Opioid_Data',
            experiment_no='1')

[Liquidten]

raise TalosReturnError("Make sure that input model returns 'out, model' where out is history object from model.fit()")

talos.utils.exceptions.TalosReturnError: Make sure that input model returns 'out, model' where out is history object from model.fit()

So i edited somethings now i am getting this error!

[DrappierTechnologies]

@Liquidten it be great if you could explain what you edited as have also encountered the AttributeError: 'Series' object has no attribute 'flatten' error. Since all my inputs were working fine for training the Keras model I find it strange that talos would throw an error on the input data.

[mikkokotila]

Very sorry for not coming back to this. Thanks a lot for all the inputs also. @DrappierTechnologies can you also share your model, Scan() command, param dictionary and indication for your input data.

[DrappierTechnologies]

Here you go and thanks for the prompt response:

def optimize_neural_hyperparameters(x_train, y_train, x_val, y_val, params):
    model = Sequential()
    model.add(Dense(params['first_neuron'], activation=params['activation'], input_shape=(x_train.shape[1],)))
    model.add(Dropout(params['dropout']))
    model.add(Dense(params['dense_neuron'], activation=params['activation']))
    model.add(Dropout(params['dropout']))
    model.add(Dense(params['dense_neuron'], activation=params['activation']))
    model.add(Dropout(params['dropout']))
    model.add(Dense(params['dense_neuron'], activation=params['activation']))
    model.add(Dropout(params['dropout']))
    model.add(Dense(params['dense_neuron'], activation=params['activation']))
    model.add(Dropout(params['dropout']))
    model.add(Dense(1, activation=params['last_activation']))
    model.compile(optimizer=params['optimizer'](lr=lr_normalizer(params['learning_rate'], params['optimizer'])),
                  loss=params['loss'],
                  metrics=params['metrics'])

    out = model.fit(x_train, y_train, 
                    epochs=params['epochs'],
                    batch_size=params['batch_size'],
                    validation_data=[x_val, y_val],
                    verbose=0,
                    callbacks=early_stopper(params['epochs'], mode='strict'))
    return out, model

neural_params = {'learning_rate': [0.01, 0.1, 0.5, 1, 5, 10],
                 'first_neuron':[4, 8, 16, 32, 64, 128, 256, 512, 1028],
                 'dense_neuron':[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1028],
                 'hidden_layers':[0, 1, 2],
                 'batch_size': [2, 4, 8, 16, 32],
                 'epochs': [100, 150, 200, 250],
                 'dropout': [0, 0.05, 0.10, 0.15, 0.25, 0.5, 5],
                 'optimizer': [Adam, Nadam, RMSprop],
                 'loss': [mean_absolute_percentage_error, mean_squared_error, mean_absolute_error],
                 'activation':[relu, elu],
                 'last_activation': [sigmoid],
                 'metrics': [['mse']]}

neural_optimized = x_train,
                           y_train.values,
                           model=optimize_neural_hyperparameters,
                           grid_downsample=.1, 
                           params=neural_params,
                           dataset_name='SAM140',
                           experiment_no='1',
                           functional_model=True)

x_train =
array([[-1.21834996,  2.11137203,  0.1717873 , ...,  1.        ,
         0.        ,  0.        ],
       [ 0.53275955, -0.03995387, -0.06867266, ...,  0.        ,
         0.        ,  0.        ],
       [ 2.52616095, -0.86046503, -0.36924761, ...,  0.        ,
         0.        ,  0.        ],
       ...,
       [-0.61262026, -0.17813704, -0.30913262, ...,  1.        ,
         0.        ,  0.        ],
       [-0.24918243,  2.2863132 , -0.30913262, ...,  1.        ,
         0.        ,  0.        ],
       [-1.21834996,  1.61724685,  1.07351214, ...,  0.        ,
         0.        ,  0.        ]])

y_train.values =
array([0.23764548, 0.41328226, 0.42270895, ..., 0.27304594, 0.26421755,
       0.27984495])

Listing all this out has helped me visualize the issue and I seem to think I have found the cause. The issue is indeed with talos as talos doesn't seem to be able to flatten the pandas Series object y_train. Sending y_train as y_train.values flattens the Series to a 1D ndarray which talos is happy accepting. Do you know if passing a pandas Series is in fact not supported by talos or could this be that I have an older version of the library loaded?

In any case now I'm receiving this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\utils\logging.py in dict_tostr(self, d)
     38     try:
---> 39         s += ',' + str(round(self._val_score, 3))
     40     except TypeError:

TypeError: type str doesn't define __round__ method

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-83-f4548f4d91a9> in <module>()
      6                            dataset_name='SAM140',
      7                            experiment_no='1',
----> 8                            functional_model=True)

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\scan\Scan.py in __init__(self, x, y, params, model, dataset_name, experiment_no, x_val, y_val, val_split, shuffle, round_limit, grid_downsample, random_method, seed, search_method, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, reduce_loss, last_epoch_value, talos_log_name, clear_tf_session, functional_model, disable_progress_bar, print_params, debug)
    164         # input parameters section ends
    165 
--> 166         self._null = self.runtime()
    167 
    168     def runtime(self):

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\scan\Scan.py in runtime(self)
    169 
    170         self = scan_prepare(self)
--> 171         self = scan_run(self)

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\scan\scan_run.py in scan_run(self)
     19                      disable=self.disable_progress_bar)
     20     while len(self.param_log) != 0:
---> 21         self = scan_round(self)
     22         self.pbar.update(1)
     23     self.pbar.close()

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\scan\scan_round.py in scan_round(self)
     46     _hr_out = run_round_results(self, _hr_out)
     47     self._val_score = get_score(self)
---> 48     write_log(self)
     49     self.result.append(_hr_out)
     50     save_result(self)

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\utils\logging.py in write_log(self)
     55 
     56     _wt_out = clean_dict(self)
---> 57     _wt_out = dict_tostr(self, _wt_out)
     58     self.logfile.write(_wt_out + '\n')
     59 

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Anaconda3_64\lib\site-packages\talos\utils\logging.py in dict_tostr(self, d)
     42         s += ',' + str(self._round_epochs)
     43         s += ',' + self.shape
---> 44         s += ',' + str(self._y_range)
     45         s += ',' + self._y_format
     46         s += ',' + str(self.val_split)

AttributeError: 'Scan' object has no attribute '_y_range'

I'd be happy to create a separate issue so that you can close this one.

[mikkokotila]

Sorry guys, this slipped through the cracks.

@Liquidten Did you end up getting things working?

@DrappierTechnologies yes, these two are separate issues, could you post a new one with yours.

In the meantime, I also suggest installing the current dev or daily-dev versions which are v.0.4.6 as many things have been improved in terms of how error-prone Talos is.

[mikkokotila]

I believe that these issues are resolved in the latest versions (> v0.4.4). Closing here. Feel free to reopen / create new issue if anything.

[Albmargar1]

Hi,

Sorry for reopening this issue, but I'm finding the same error. I was trying to replicate the Comprehensive example when the error popped-up. My talos version is 0.4.5.

Here is the complete error:

File "<ipython-input-1-705d4f52734c>", line 1, in <module>
    runfile('C:/Users/Alberto/Documents/Work/Neural Network/talos_simple.py', wdir='C:/Users/Alberto/Documents/Work/Neural Network')

  File "C:\Users\Alberto\Anaconda3\envs\tensorflow\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 704, in runfile
    execfile(filename, namespace)

  File "C:\Users\Alberto\Anaconda3\envs\tensorflow\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/Alberto/Documents/Work/Neural Network/talos_simple.py", line 54, in <module>
    grid_downsample=.01)

  File "C:\Users\Alberto\Anaconda3\envs\tensorflow\lib\site-packages\talos\scan\Scan.py", line 163, in __init__
    self._null = self.runtime()

  File "C:\Users\Alberto\Anaconda3\envs\tensorflow\lib\site-packages\talos\scan\Scan.py", line 168, in runtime
    self = scan_run(self)

  File "C:\Users\Alberto\Anaconda3\envs\tensorflow\lib\site-packages\talos\scan\scan_run.py", line 20, in scan_run
    self = scan_round(self)

  File "C:\Users\Alberto\Anaconda3\envs\tensorflow\lib\site-packages\talos\scan\scan_round.py", line 37, in scan_round
    raise TalosReturnError("Make sure that input model returns 'out, model' where out is history object from model.fit()")

TalosReturnError: Make sure that input model returns 'out, model' where out is history object from model.fit()

[mikkokotila]

This is resolved in #185. Since v.0.4.7 the notebook is fixed.

[jberglinds]

I had the same issue as OP.
The problem came from using Pandas to process my data and then sending in the Pandas objects to the Scan function.

A Pandas DataFrame or Series contains a raw numpy array in the values-property. When using that as input to Scan I got rid of my problems.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.values.html

I believe Keras does this automatically if you pass a Pandas object.

[mikkokotila]

If your data is in the form some_series then just pass some_series.values instead, which will yield a Numpy array instead.

[jberglinds]

Yes, that's what I did.

But if talos is meant to be used with Keras models, and Keras unwraps Pandas objects automatically, then maybe talos should do the same thing. Just a thought. Since Pandas is widely used, I don't think we're the first ones to encounter this error.