New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'ProgbarLogger' object has no attribute 'log_values' #3657

Closed
kevkid opened this Issue Sep 1, 2016 · 23 comments

Comments

Projects
None yet
@kevkid
Copy link

kevkid commented Sep 1, 2016

Please make sure that the boxes below are checked before you submit your issue. Thank you!

  • Check that you are up-to-date with the master branch of Keras. You can update with:
    pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
  • If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
    pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
  • Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I am performing batch learning and after a few batches I get this error from this line of code:

model.fit(Xtrain, Ytrain, batch_size=128, nb_epoch=1,
                  verbose=1,validation_split=0.01,
                  callbacks=[ModelCheckpoint(weightStr, monitor='val_loss', verbose=0, save_best_only=True, mode='auto')])

Traceback (most recent call last):

  File "<ipython-input-1-0ab90ed05873>", line 321, in <module>
    callbacks=[ModelCheckpoint(weightStr, monitor='val_loss', verbose=0, save_best_only=True, mode='auto')])

  File "/home/kevin/.local/lib/python2.7/site-packages/keras/models.py", line 620, in fit
    sample_weight=sample_weight)

  File "/home/kevin/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1104, in fit
    callback_metrics=callback_metrics)

  File "/home/kevin/.local/lib/python2.7/site-packages/keras/engine/training.py", line 842, in _fit_loop
    callbacks.on_epoch_end(epoch, epoch_logs)

  File "/home/kevin/.local/lib/python2.7/site-packages/keras/callbacks.py", line 40, in on_epoch_end
    callback.on_epoch_end(epoch, logs)

  File "/home/kevin/.local/lib/python2.7/site-packages/keras/callbacks.py", line 196, in on_epoch_end
    self.progbar.update(self.seen, self.log_values, force=True)

AttributeError: 'ProgbarLogger' object has no attribute 'log_values'

I have no idea why I get this error, It seems to happen randomly, can anyone point me in the right direction?

Here is the module of code that I am running:

for e in range(numEpoch):
    numOfImgToLoad = 50000#we can tune this
    totalNumberOfImages = len(imagesAndClass)
    runningTotal = 0
    startingPoint = 0
    endingPoint = numOfImgToLoad
    while totalNumberOfImages > 0:
        print "StartingPoint: {}, endingPoint {}".format(startingPoint, endingPoint)
        totalNumberOfImages = totalNumberOfImages - numOfImgToLoad#subtract the number of images loaded into mem
        if totalNumberOfImages < 0:
            remainder = totalNumberOfImages + numOfImgToLoad
            (Xtrain, Ytrain) = loadImages(imagesAndClass[startingPoint:remainder])
            Xtrain = np.array(Xtrain).reshape(len(Xtrain), 1, 106, 106)#np.array(Xtrain).reshape(4415, 1, 106, 106)
            runningTotal += remainder
        else:
            (Xtrain, Ytrain) = loadImages(imagesAndClass[startingPoint:endingPoint])
            Xtrain = np.array(Xtrain).reshape(len(Xtrain), 1, 106, 106)
            runningTotal += numOfImgToLoad
            startingPoint = endingPoint+1
            endingPoint = startingPoint + numOfImgToLoad - 1

        Xtrain /= 255#change pixel value to between 0 and 1
        Xtrain = Xtrain.astype('float32')
        Ytrain = np_utils.to_categorical(Ytrain, len(classes)+1)
        Ytrain = np.array(Ytrain)
        print "Starting epoch {}".format(e)
        model.fit(Xtrain, Ytrain, batch_size=128, nb_epoch=1,
                  verbose=1,validation_split=0.01,
                  callbacks=[ModelCheckpoint(weightStr, monitor='val_loss', verbose=0, save_best_only=True, mode='auto')])
                  #callbacks=[EarlyStopping(monitor='val_loss', patience=2)])
        #print "Starting epoch {} on image {}".format(e, runningTotal)
        print "Killing Xtrain and resetting"
        del Xtrain
        del Ytrain
@fchollet

This comment has been minimized.

Copy link
Collaborator

fchollet commented Sep 1, 2016

Can you post a minimal, short, standalone script to reproduce your issue?

@kevkid

This comment has been minimized.

Copy link

kevkid commented Sep 2, 2016

Thanks fchollet, I found the bug in my code.
Apparently this line was causing it:
(Xtrain, Ytrain) = loadImages(imagesAndClass[startingPoint:remainder])

It should have been

(Xtrain, Ytrain) = loadImages(imagesAndClass[startingPoint:startingPoint+remainder])

It was trying to return an array with an invalid range something like: 500000-4900

@kevkid kevkid closed this Sep 2, 2016

@SaadBinShahid

This comment has been minimized.

Copy link

SaadBinShahid commented Feb 8, 2017

Hi,

I've the same error. I'm following this https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html tutorial for Using pre-trained word embeddings in a Keras model.

  File "/home/mrx/src/experpk/exper.pk/API/Product/management/commands/categorize.py", line 130, in handle
    model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=2, batch_size=50, verbose=1)
  File "/home/mrx/src/experpk/venv-experpk/local/lib/python2.7/site-packages/keras/engine/training.py", line 1192, in fit
    initial_epoch=initial_epoch)
  File "/home/mrx/src/experpk/venv-experpk/local/lib/python2.7/site-packages/keras/engine/training.py", line 912, in _fit_loop
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/home/mrx/src/experpk/venv-experpk/local/lib/python2.7/site-packages/keras/callbacks.py", line 76, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/home/mrx/src/experpk/venv-experpk/local/lib/python2.7/site-packages/keras/callbacks.py", line 265, in on_epoch_end
    self.progbar.update(self.seen, self.log_values, force=True)
AttributeError: 'ProgbarLogger' object has no attribute 'log_values'

Respective code is ;

        if self.verbose:
            # here
            self.progbar.update(self.seen, self.log_values, force=True)

Can you please help me.

@vladbph

This comment has been minimized.

Copy link

vladbph commented May 6, 2017

For method fit() Keras should throw an exception when using validation_split and training set is ended up to be 0... Example: your last batch in epoch contains 1 sample and validation_split = 0.3 => training size = 0...
When training set size is 0 you will see this exception...
If you look at callbacks.py in keras, class ProgbarLogger, log_values member is created ONLY when batch starts... So if you don't have data in the batch it gets to on_epoch_end with log_values undefined...

@oswaldoludwig

This comment has been minimized.

Copy link
Contributor

oswaldoludwig commented Jun 3, 2017

It seems this is not solved yet. I uploaded Keras yesterday (2/6/2017) and the code is still raising this message: AttributeError: 'ProgbarLogger' object has no attribute 'log_values'

@sixhobbits

This comment has been minimized.

Copy link

sixhobbits commented Jun 18, 2017

from keras.models import Sequential
from keras.layers import Embedding, GRU, Dense
import numpy as np


def get_model():
    model = Sequential()
    model.add(Embedding(input_dim=27, output_dim=300))
    model.add(GRU(256))
    model.add(Dense(27, activation='softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam')
    return model

model = get_model()
X = np.zeros((0, 10), dtype=np.int)
y = np.zeros((0, 27), dtype=np.bool)
model.fit(X, y)

Here's a self-contained example that reproduces the error. @fchollet , can we reopen this?

Here's the output I get

~ g$ python3 keras_error.py
Using TensorFlow backend.
Epoch 1/10
Traceback (most recent call last):
  File "keras_error.py", line 17, in <module>
    model.fit(X, y)
  File "/usr/local/lib/python3.6/site-packages/keras/models.py", line 870, in fit
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python3.6/site-packages/keras/engine/training.py", line 1507, in fit
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python3.6/site-packages/keras/engine/training.py", line 1176, in _fit_loop
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "/usr/local/lib/python3.6/site-packages/keras/callbacks.py", line 77, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "/usr/local/lib/python3.6/site-packages/keras/callbacks.py", line 309, in on_epoch_end
    self.progbar.update(self.seen, self.log_values, force=True)
AttributeError: 'ProgbarLogger' object has no attribute 'log_values'
@saransh-mehta

This comment has been minimized.

Copy link

saransh-mehta commented Jun 22, 2017

I'm also getting the same error while training a four layer LSTM on small conversation corpus.epochs = 500
epochs = 500
for i in range(10):
model.fit(vecX, vecY, epochs = epochs, validation_split = 0.2, verbose = 1)
model.save('LSTM' + str((epochs * i + 500)) + '.h5')

this gives out

Train on 0 samples, validate on 1 samples
Epoch 1/500


AttributeError Traceback (most recent call last)
in ()
1 epochs = 500
2 for i in range(10):
----> 3 model.fit(vecX, vecY, epochs = epochs, validation_split = 0.2, verbose = 1)
4 model.save('LSTM' + str((epochs * i + 500)) + '.h5')

~/.local/lib/python3.5/site-packages/keras/models.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, **kwargs)
868 class_weight=class_weight,
869 sample_weight=sample_weight,
--> 870 initial_epoch=initial_epoch)
871
872 def evaluate(self, x, y, batch_size=32, verbose=1,

~/.local/lib/python3.5/site-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, **kwargs)
1505 val_f=val_f, val_ins=val_ins, shuffle=shuffle,
1506 callback_metrics=callback_metrics,
-> 1507 initial_epoch=initial_epoch)
1508
1509 def evaluate(self, x, y, batch_size=32, verbose=1, sample_weight=None):

~/.local/lib/python3.5/site-packages/keras/engine/training.py in fit_loop(self, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch)
1174 for l, o in zip(out_labels, val_outs):
1175 epoch_logs['val
' + l] = o
-> 1176 callbacks.on_epoch_end(epoch, epoch_logs)
1177 if callback_model.stop_training:
1178 break

~/.local/lib/python3.5/site-packages/keras/callbacks.py in on_epoch_end(self, epoch, logs)
75 logs = logs or {}
76 for callback in self.callbacks:
---> 77 callback.on_epoch_end(epoch, logs)
78
79 def on_batch_begin(self, batch, logs=None):

~/.local/lib/python3.5/site-packages/keras/callbacks.py in on_epoch_end(self, epoch, logs)
307 self.log_values.append((k, logs[k]))
308 if self.verbose:
--> 309 self.progbar.update(self.seen, self.log_values, force=True)
310
311

AttributeError: 'ProgbarLogger' object has no attribute 'log_values'

@ss32

This comment has been minimized.

Copy link

ss32 commented Jul 9, 2017

Same error, and I tried removing validation_split from my fit command, but no change.

@saransh-mehta

This comment has been minimized.

Copy link

saransh-mehta commented Jul 9, 2017

Have you checked the dimensions for both ur x and y for train and test, I had a dimension issue with my y train. Generally the error arising if u tey to split data of incorrect size , say if it's 1, and u try to split it, it gives this error

@ultrarelativistic

This comment has been minimized.

Copy link

ultrarelativistic commented Jul 17, 2017

Ran into this issue today. saransh-mehta suggestion was the solution. Also ran into an issue where the number of samples for test/train split was not sufficient. train was 0 samples.

@fchollet

This comment has been minimized.

Copy link
Collaborator

fchollet commented Jul 18, 2017

If this is a common issue, we need a clear error message to handle it. Please open a PR to add appropriate error handling.

@easyfly007

This comment has been minimized.

Copy link

easyfly007 commented Sep 2, 2017

Hi Guys,
I also meet this issue, but it happened that the sample size is small, e.g., I used a sample size 2 and do model.fit it by split ratio 0.2 then get this error. but when I used sample size >1000, then it disappeared.

hope this information may help.

regards, Yifei

@mihaha

This comment has been minimized.

Copy link

mihaha commented Oct 10, 2017

I also encountered this error today. Had to set verbose=0 to fix.
Please reopen the issue. It needs to be fixed!

@akaszynski

This comment has been minimized.

Copy link

akaszynski commented Nov 5, 2017

Encountered this error when my training set was an empty array. Using Keras 2.0.9.

A more descriptive error message might be helpful

@abiro

This comment has been minimized.

Copy link

abiro commented Jan 25, 2018

This happens if steps_per_epoch is 0. Make sure that your batch size is not greater than the dataset size to avoid it.

@Ather23

This comment has been minimized.

Copy link

Ather23 commented Apr 7, 2018

I got a similar issue when my training data set was empty after data cleaning and normalization. Might be worth checking you data set size before training.

@msollami

This comment has been minimized.

Copy link

msollami commented Apr 24, 2018

I've hit this error in Keras 2.1.5 as well.

@1179548504

This comment has been minimized.

Copy link

1179548504 commented May 7, 2018

I also encountered this problem and I do not know how to solve it

@zkeenly

This comment has been minimized.

Copy link

zkeenly commented May 26, 2018

maybe this issue occurrence due to less train sample.increase more train sample can solve it

@bzamecnik

This comment has been minimized.

Copy link
Contributor

bzamecnik commented May 28, 2018

Encountered this in 2.1.6 when supplying empty training set by mistake. IMHO the code should handle this gracefully.

@Anoopparjanya

This comment has been minimized.

Copy link

Anoopparjanya commented Jun 19, 2018

@kevkid @oswaldoludwig
Attached is my gist. https://gist.github.com/Anoopparjanya/d921e3edf1ef9eeb8621ed6138f9557f/revisions

Im getting the same error though i have installed updated versions f keras and Theano mentioned above.

Please help me out,how to overcome this issue?

@ebuildy

This comment has been minimized.

Copy link

ebuildy commented Jun 19, 2018

Set verbose=0:

model.fit(X, Y, verbose=0)

Or make sure your dataset is bigger than the batch size.

@ZhangXiaoying0116

This comment has been minimized.

Copy link

ZhangXiaoying0116 commented Oct 1, 2018

将batch_size设置为较小的数字。当您的batch_size设置为比样本集大小更大的值时,会出现此错误。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment