run main.py with error #10

hanyaqian · 2017-05-25T04:00:33Z

[25/05/2017 11:57:37] Training parameters: {'reload_epoch': 0, 'shuffle': True, 'verbose': 1, 'data_augmentation': False, 'class_weights': None, 'epochs_for_save': 1, 'lr_gamma': 0.8, 'start_eval_on_epoch': 1, 'n_epochs': 500, 'patience': 20, 'mean_substraction': True, 'normalize': False, 'n_parallel_loaders': 1, 'each_n_epochs': 1, 'patience_check_split': 'val', 'lr_decay': None, 'homogeneous_batches': False, 'batch_size': 50, 'metric_check': 'Bleu_4', 'num_iterations_val': None, 'epoch_offset': 0, 'joint_batches': 4, 'maxlen': 50, 'eval_on_epochs': True, 'eval_on_sets': [], 'extra_callbacks': [<keras_wrapper.extra.callbacks.EvalPerformance object at 0x7fc1dfd6fad0>, <keras_wrapper.extra.callbacks.Sample object at 0x7fc1e5723ed0>]}
ERROR (theano.gof.opt): SeqOptimizer apply <theano.scan_module.scan_opt.PushOutScanOutput object at 0x7fc216a1a110>
[25/05/2017 11:57:41] SeqOptimizer apply <theano.scan_module.scan_opt.PushOutScanOutput object at 0x7fc216a1a110>
ERROR (theano.gof.opt): Traceback:
[25/05/2017 11:57:41] Traceback:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 235, in apply
sub_prof = optimizer.optimize(fgraph)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 87, in optimize
ret = self.apply(fgraph, *args, **kwargs)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 685, in apply
node = self.process_node(fgraph, node)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 745, in process_node
node, args)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 854, in push_out_inner_vars
add_as_nitsots)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 906, in add_nitsot_outputs
reason='scanOp_pushout_output')
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 391, in replace_all_validate_remove
chk = fgraph.replace_all_validate(replacements, reason)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 365, in replace_all_validate
fgraph.validate()
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 256, in validate_
ret = fgraph.execute_callbacks('validate')
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/fg.py", line 589, in execute_callbacks
fn(self, *args, **kwargs)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 422, in validate
raise theano.gof.InconsistencyError("Trying to reintroduce a removed node")
InconsistencyError: Trying to reintroduce a removed node

[25/05/2017 11:57:41] Traceback (most recent call last):
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 235, in apply
sub_prof = optimizer.optimize(fgraph)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 87, in optimize
ret = self.apply(fgraph, *args, **kwargs)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 685, in apply
node = self.process_node(fgraph, node)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 745, in process_node
node, args)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 854, in push_out_inner_vars
add_as_nitsots)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 906, in add_nitsot_outputs
reason='scanOp_pushout_output')
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 391, in replace_all_validate_remove
chk = fgraph.replace_all_validate(replacements, reason)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 365, in replace_all_validate
fgraph.validate()
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 256, in validate_
ret = fgraph.execute_callbacks('validate')
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/fg.py", line 589, in execute_callbacks
fn(self, *args, **kwargs)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 422, in validate
raise theano.gof.InconsistencyError("Trying to reintroduce a removed node")
InconsistencyError: Trying to reintroduce a removed node

Epoch 1/500
9850/9900 [============================>.] - ETA: 0s - loss: 2.4758[25/05/2017 12:00:49]
<<< Predicting outputs of val set >>>
Sampling 100/100 - ETA: 0s
Total cost of the translations: 203.730323 Average cost of the translations: 2.037303
The sampling took: 28.331956 secs (Speed: 0.283320 sec/sample)
[25/05/2017 12:01:17] Decoding beam search prediction ...
[25/05/2017 12:01:17] Evaluating on metric coco
bash: tercom.7.25: No such file or directory
Traceback (most recent call last):
File "main.py", line 370, in
train_model(parameters, args.dataset)
File "main.py", line 131, in train_model
nmt_model.trainNet(dataset, training_params)
File "build/bdist.linux-x86_64/egg/keras_wrapper/cnn_model.py", line 702, in trainNet
File "build/bdist.linux-x86_64/egg/keras_wrapper/cnn_model.py", line 864, in __train
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/engine/training.py", line 1623, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/callbacks.py", line 43, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "build/bdist.linux-x86_64/egg/keras_wrapper/extra/callbacks.py", line 185, in on_epoch_end
File "build/bdist.linux-x86_64/egg/keras_wrapper/extra/callbacks.py", line 320, in evaluate
File "build/bdist.linux-x86_64/egg/keras_wrapper/extra/evaluation.py", line 69, in get_coco_score
File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Coco_Caption-0.0-py2.7.egg/pycocoevalcap/ter/ter.py", line 55, in compute_score
return float(score), None
ValueError: could not convert string to float:

Can you tell me some detial about the error? Thanks very much!

lvapeab · 2017-05-25T07:30:14Z

Hello,
thanks for your interest.

It seems that you don't have the jar file that computes the TER metric. Have you installed the Coco-caption package from my fork?

If so, please, can you attach the output of:

ls /home/hanyaqian/anaconda2/lib/python2.7/site-packages/Coco_Caption-0.0-py2.7.egg/pycocoevalcap/ter/

Regards

hanyaqian · 2017-05-25T08:14:39Z

Hi @lvapeab , i hava installed the Coco-caption package successfully.
__init__.py __init__.pyc ter.py ter.pyc

lvapeab · 2017-05-25T08:38:18Z

You lack the tercom file. I don't know if there was a problem in the installation.

You can try reinstalling it again or manually placing the tercom.7.25 file in your /home/hanyaqian/anaconda2/lib/python2.7/site-packages/Coco_Caption-0.0-py2.7.egg/pycocoevalcap/ter/ folder.

hanyaqian · 2017-05-25T09:18:30Z

I reinstalling Coco-caption by the command of python setup.py install, and still lack the tercom file, then manually placing the tercom.7.25 file in the above folder.
And happen another error:
[25/05/2017 17:05:48] Evaluating on metric coco Traceback (most recent call last): File "main.py", line 373, in <module> train_model(parameters, args.dataset) File "main.py", line 133, in train_model nmt_model.trainNet(dataset, training_params) File "build/bdist.linux-x86_64/egg/keras_wrapper/cnn_model.py", line 702, in trainNet File "build/bdist.linux-x86_64/egg/keras_wrapper/cnn_model.py", line 864, in __train File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/engine/training.py", line 1623, in fit_generator callbacks.on_epoch_end(epoch, epoch_logs) File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Keras-1.2.0-py2.7.egg/keras/callbacks.py", line 43, in on_epoch_end callback.on_epoch_end(epoch, logs) File "build/bdist.linux-x86_64/egg/keras_wrapper/extra/callbacks.py", line 185, in on_epoch_end File "build/bdist.linux-x86_64/egg/keras_wrapper/extra/callbacks.py", line 320, in evaluate File "build/bdist.linux-x86_64/egg/keras_wrapper/extra/evaluation.py", line 69, in get_coco_score File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Coco_Caption-0.0-py2.7.egg/pycocoevalcap/meteor/meteor.py", line 35, in compute_score score_line, stat = self._stat(res[i][0], gts[i]) File "/home/hanyaqian/anaconda2/lib/python2.7/site-packages/Coco_Caption-0.0-py2.7.egg/pycocoevalcap/meteor/meteor.py", line 54, in _stat self.meteor_p.stdin.write('{}\n'.format(score_line)) IOError: [Errno 32] Broken pipe

under the folder of /home/hanyaqian/anaconda2/lib/python2.7/site-packages/Coco_Caption-0.0-py2.7.egg/pycocoevalcap/meteor is __init__.py __init__.pyc meteor-1.5.jar meteor.py meteor.pyc

the meteor-1.5.jar is also lacked while installed the Coco-caption.

lvapeab · 2017-05-25T09:57:40Z

Ok, I've seen the mistake and corrected it in the Coco-caption package (lvapeab/coco-caption@ca089be).

Pull the coco-caption package an run again the
python setup.py install

hanyaqian · 2017-05-25T10:49:07Z

Hi @lvapeab , it works, thank you very much!

yanmengqi · 2018-03-12T11:30:50Z

请问你是在中英翻译吗？这个bug你解决了吗？

qazs · 2018-08-27T11:09:06Z

Hi @lvapeab I'm still getting the error, is it because I'm using windows?

Traceback (most recent call last):
  File "main.py", line 445, in <module>
    train_model(parameters, args.dataset)
  File "main.py", line 169, in train_model
    nmt_model.trainNet(dataset, training_params)
  File "d:\docker\ai\nmt-keras\src\keras-wrapper\keras_wrapper\cnn_model.py", line 832, in trainNet
    self.__train(ds, params)
  File "d:\docker\ai\nmt-keras\src\keras-wrapper\keras_wrapper\cnn_model.py", line 1107, in __train
    initial_epoch=params['epoch_offset'])
  File "d:\docker\ai\nmt-keras\src\keras\keras\legacy\interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "d:\docker\ai\nmt-keras\src\keras\keras\engine\training.py", line 1485, in fit_generator
    initial_epoch=initial_epoch)
  File "d:\docker\ai\nmt-keras\src\keras\keras\engine\training_generator.py", line 247, in fit_generator
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "d:\docker\ai\nmt-keras\src\keras\keras\callbacks.py", line 77, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "d:\docker\ai\nmt-keras\src\keras-wrapper\keras_wrapper\extra\callbacks.py", line 277, in on_epoch_end
    self.evaluate(epoch, counter_name='epoch')
  File "d:\docker\ai\nmt-keras\src\keras-wrapper\keras_wrapper\extra\callbacks.py", line 486, in evaluate
    split=s)
  File "d:\docker\ai\nmt-keras\src\keras-wrapper\keras_wrapper\extra\evaluation.py", line 60, in get_coco_score
    score, _ = scorer.compute_score(refs, hypo)
  File "d:\docker\ai\nmt-keras\src\coco-caption\pycocoevalcap\meteor\meteor.py", line 40, in compute_score
    stat = self._stat(res[i][0], gts[i])
  File "d:\docker\ai\nmt-keras\src\coco-caption\pycocoevalcap\meteor\meteor.py", line 69, in _stat
    self.meteor_p.stdin.flush()
OSError: [Errno 22] Invalid argument

lvapeab · 2018-08-27T11:14:07Z

Hi @qazs ,

do you have java (version 1.8.0 or newer) installed?

qazs · 2018-08-27T11:15:55Z

Yes, here's my java version

$ java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) Client VM (build 25.144-b01, mixed mode)
(nmt-keras)

lvapeab · 2018-08-27T11:30:38Z

Hmm... It seems this error has also been reported in the original scoring library (tylin/coco-caption#28). Unfortunately, I cannot test it in Windows.

As a workaround, I suggest you to edit your keras_wrapper\extra\evaluation.py file (in your case, located in d:\docker\ai\nmt-keras\src\keras-wrapper\keras_wrapper\extra\evaluation.py) and comment the lines which call METEOR 56-57:

# if extra_vars.get('language', 'en') in accepted_langs:
# scorers.append((Meteor(language=extra_vars['language']), "METEOR"))

Moreover, if there is a problem with subprocess and Windows, it is possible that the TER score won't also work. If this is the case, comment the corresponding line (in the same file):

 # (Ter(), "TER"),

qazs · 2018-08-27T12:00:28Z

Commenting out the lines works, thanks!

hanyaqian closed this as completed May 25, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

run main.py with error #10

run main.py with error #10

hanyaqian commented May 25, 2017 •

edited

lvapeab commented May 25, 2017

hanyaqian commented May 25, 2017 •

edited

lvapeab commented May 25, 2017 •

edited

hanyaqian commented May 25, 2017 •

edited

lvapeab commented May 25, 2017 •

edited

hanyaqian commented May 25, 2017

yanmengqi commented Mar 12, 2018

qazs commented Aug 27, 2018

lvapeab commented Aug 27, 2018

qazs commented Aug 27, 2018

lvapeab commented Aug 27, 2018

qazs commented Aug 27, 2018

run main.py with error #10

run main.py with error #10

Comments

hanyaqian commented May 25, 2017 • edited

lvapeab commented May 25, 2017

hanyaqian commented May 25, 2017 • edited

lvapeab commented May 25, 2017 • edited

hanyaqian commented May 25, 2017 • edited

lvapeab commented May 25, 2017 • edited

hanyaqian commented May 25, 2017

yanmengqi commented Mar 12, 2018

qazs commented Aug 27, 2018

lvapeab commented Aug 27, 2018

qazs commented Aug 27, 2018

lvapeab commented Aug 27, 2018

qazs commented Aug 27, 2018

hanyaqian commented May 25, 2017 •

edited

hanyaqian commented May 25, 2017 •

edited

lvapeab commented May 25, 2017 •

edited

hanyaqian commented May 25, 2017 •

edited

lvapeab commented May 25, 2017 •

edited