fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX. #14

wenwei202 · 2017-05-17T19:13:41Z

By running ./quick-start-cbt-ne.sh, what might be the issue?

systems Ubuntu 16.04 with python and theano of

$ python
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import theano as th
Using cuDNN version 5110 on context None
Mapped name None to device cuda: TITAN X (Pascal) (0000:22:00.0)
>>> th.__version__
'0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291'
>>>

$ ./quick-start-cbt-ne.sh 
+++ DATA_ROOT=../data/CBTest/data/
+++ TRAIN=cbtest_NE_train.txt
+++ VALID=cbtest_NE_valid_2000ex.txt
+++ TEST=cbtest_NE_test_2500ex.txt
+++ OUT_DIR=test_output_cbt
+++ DATASET_TYPE=cbt
+++ for HYPERPARAMETERS in '"${@:7}"'
+++ LOG_FILE=cbtest_NE_train.txt.-ehd_128_-sed_256.log
+++ mkdir test_output_cbt -p
+++ THEANO_FLAGS=floatX=float32,device=gpu
+++ PYTHONPATH=~/github/::./quick-start-generic.sh
+++ python text_comprehension/as_reader.py -b 32 -ehd 128 -sed 256 -lr 0.001 -p 2 --dataset_root ../data/CBTest/data/ --train cbtest_NE_train.txt --valid cbtest_NE_valid_2000ex.txt --test cbtest_NE_test_2500ex.txt --output_dir test_output_cbt --evaluate_every_n 1e --patience_metric accuracy --own_eval --append_metaparams --no_html --dataset_type cbt --save_path test_output_cbt/model.blocks.pkl
+++ tee test_output_cbt/cbtest_NE_train.txt.-ehd_128_-sed_256.log -i
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
 https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: TITAN X (Pascal) (CNMeM is disabled, cuDNN 5110)
Traceback (most recent call last):
  File "text_comprehension/as_reader.py", line 166, in <module>
Output will be stored in test_output_cbt
Computing new vocabulary for file ../data/CBTest/data/cbtest_NE_train.txt.
Processed line 100000
Processed line 200000
Processed line 300000
Processed line 400000
Processed line 500000
Processed line 600000
Processed line 700000
Processed line 800000
Processed line 900000
Processed line 1000000
Processed line 1100000
Processed line 1200000
Processed line 1300000
Processed line 1400000
Processed line 1500000
Processed line 1600000
Processed line 1700000
Processed line 1800000
Processed line 1900000
Processed line 2000000
Processed line 2100000
Processed line 2200000
Processed line 2300000
STATISTICS
Total words: 53825761
Total distinct words: 60278
    exp.execute()
  File "~/github/asreader/asreader/text_comprehension/text_comprehension_base.py", line 343, in execute
    read_ahead=self.args.sort_k_batches_ahead)
  File "~/github/asreader/asreader/text_comprehension/text_comprehension_base.py", line 307, in get_stream
    stream = Padding(stream, mask_sources=['context', 'question', 'candidates'])
  File "~/.local/lib/python2.7/site-packages/fuel/transformers/__init__.py", line 703, in __init__
    self.mask_dtype = config.floatX
  File "~/.local/lib/python2.7/site-packages/fuel/config_parser.py", line 150, in __getattr__
    "provided: {}.".format(key))
fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX.
+++ for HYPERPARAMETERS in '"${@:7}"'
+++ LOG_FILE=cbtest_NE_train.txt.-ehd_384_-sed_384.log
+++ mkdir test_output_cbt -p
+++ THEANO_FLAGS=floatX=float32,device=gpu
+++ tee test_output_cbt/cbtest_NE_train.txt.-ehd_384_-sed_384.log -i
+++ PYTHONPATH=/tmp::./quick-start-generic.sh
+++ python text_comprehension/as_reader.py -b 32 -ehd 384 -sed 384 -lr 0.001 -p 2 --dataset_root ../data/CBTest/data/ --train cbtest_NE_train.txt --valid cbtest_NE_valid_2000ex.txt --test cbtest_NE_test_2500ex.txt --output_dir test_output_cbt --evaluate_every_n 1e --patience_metric accuracy --own_eval --append_metaparams --no_html --dataset_type cbt --save_path test_output_cbt/model.blocks.pkl
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
 https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: TITAN X (Pascal) (CNMeM is disabled, cuDNN 5110)
Traceback (most recent call last):
  File "text_comprehension/as_reader.py", line 166, in <module>
    Output will be stored in test_output_cbt
Computing new vocabulary for file ../data/CBTest/data/cbtest_NE_train.txt.
Processed line 100000
Processed line 200000
Processed line 300000
Processed line 400000
Processed line 500000
Processed line 600000
Processed line 700000
Processed line 800000
Processed line 900000
Processed line 1000000
Processed line 1100000
Processed line 1200000
Processed line 1300000
Processed line 1400000
Processed line 1500000
Processed line 1600000
Processed line 1700000
Processed line 1800000
Processed line 1900000
Processed line 2000000
Processed line 2100000
Processed line 2200000
Processed line 2300000
STATISTICS
Total words: 53825761
Total distinct words: 60278
exp.execute()
  File "~/github/asreader/asreader/text_comprehension/text_comprehension_base.py", line 343, in execute
    read_ahead=self.args.sort_k_batches_ahead)
  File "~/github/asreader/asreader/text_comprehension/text_comprehension_base.py", line 307, in get_stream
    stream = Padding(stream, mask_sources=['context', 'question', 'candidates'])
  File "~/.local/lib/python2.7/site-packages/fuel/transformers/__init__.py", line 703, in __init__
    self.mask_dtype = config.floatX
  File "~/.local/lib/python2.7/site-packages/fuel/config_parser.py", line 150, in __getattr__
    "provided: {}.".format(key))
fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX.
+++ python text_comprehension/eval/copyBestPredictions.py -vp validation.txt. -tp test.txt. -i test_output_cbt -o test_output_cbt/best_predictions
Validation files:

Best predictions to be copied:

Best validation model:
Traceback (most recent call last):
  File "text_comprehension/eval/copyBestPredictions.py", line 124, in <module>
    print bestValModel['params']
KeyError: 'params'

The text was updated successfully, but these errors were encountered:

wenwei202 · 2017-05-17T21:08:38Z

I can skip this error by
adding floatX: float32 in ~/.fuelrc but get another issue:

$ ./quick-start-cbt-ne.sh 
+++ DATA_ROOT=../data/CBTest/data/
+++ TRAIN=cbtest_NE_train.txt
+++ VALID=cbtest_NE_valid_2000ex.txt
+++ TEST=cbtest_NE_test_2500ex.txt
+++ OUT_DIR=test_output_cbt
+++ DATASET_TYPE=cbt
+++ for HYPERPARAMETERS in '"${@:7}"'
+++ LOG_FILE=cbtest_NE_train.txt.-ehd_128_-sed_256.log
+++ mkdir test_output_cbt -p
+++ THEANO_FLAGS=floatX=float32,device=gpu
+++ PYTHONPATH=/tmp::./quick-start-generic.sh
+++ python text_comprehension/as_reader.py -b 32 -ehd 128 -sed 256 -lr 0.001 -p 2 --dataset_root ../data/CBTest/data/ --train cbtest_NE_train.txt --valid cbtest_NE_valid_2000ex.txt --test cbtest_NE_test_2500ex.txt --output_dir test_output_cbt --evaluate_every_n 1e --patience_metric accuracy --own_eval --append_metaparams --no_html --dataset_type cbt --save_path test_output_cbt/model.blocks.pkl
+++ tee test_output_cbt/cbtest_NE_train.txt.-ehd_128_-sed_256.log -i
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
 https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: TITAN X (Pascal) (CNMeM is disabled, cuDNN 5110)
ERROR (theano.gof.opt): SeqOptimizer apply <theano.scan_module.scan_opt.PushOutScanOutput object at 0x7f05131dda50>
ERROR:theano.gof.opt:SeqOptimizer apply <theano.scan_module.scan_opt.PushOutScanOutput object at 0x7f05131dda50>
ERROR (theano.gof.opt): Traceback:
ERROR:theano.gof.opt:Traceback:
ERROR (theano.gof.opt): Traceback (most recent call last):
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 235, in apply
    sub_prof = optimizer.optimize(fgraph)
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 87, in optimize
    ret = self.apply(fgraph, *args, **kwargs)
  File "~/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 685, in apply
    node = self.process_node(fgraph, node)
  File "~/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 745, in process_node
    node, args)
  File "~/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 854, in push_out_inner_vars
    add_as_nitsots)
  File "~/anaconda2/lib/python2.7/site-packages/theano/scan_module/scan_opt.py", line 906, in add_nitsot_outputs
    reason='scanOp_pushout_output')
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 391, in replace_all_validate_remove
    chk = fgraph.replace_all_validate(replacements, reason)
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 365, in replace_all_validate
    fgraph.validate()
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 256, in validate_
    ret = fgraph.execute_callbacks('validate')
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/fg.py", line 589, in execute_callbacks
    fn(self, *args, **kwargs)
  File "~/anaconda2/lib/python2.7/site-packages/theano/gof/toolbox.py", line 422, in validate
    raise theano.gof.InconsistencyError("Trying to reintroduce a removed node")
InconsistencyError: Trying to reintroduce a removed node

wenwei202 · 2017-05-17T22:06:26Z

skip raise theano.gof.InconsistencyError("Trying to reintroduce a removed node") by deleting THEANO_FLAGS in quick-start-generic.sh, and using default ones in ~/.theanorc.

#~/.theanorc
[global]
device = cuda
floatX = float32
force_device = True

But got new issue:

$ ./quick-start-cbt-ne.sh
+++ DATA_ROOT=../data/CBTest/data/
+++ TRAIN=cbtest_NE_train.txt
+++ VALID=cbtest_NE_valid_2000ex.txt
+++ TEST=cbtest_NE_test_2500ex.txt
+++ OUT_DIR=test_output_cbt
+++ DATASET_TYPE=cbt
+++ for HYPERPARAMETERS in '"${@:7}"'
+++ LOG_FILE=cbtest_NE_train.txt.-ehd_128_-sed_256.log
+++ mkdir test_output_cbt -p
+++ PYTHONPATH=/tmp::./quick-start-generic.sh
+++ python text_comprehension/as_reader.py -b 32 -ehd 128 -sed 256 -lr 0.001 -p 2 --dataset_root ../data/CBTest/data/ --train cbtest_NE_train.txt --valid cbtest_NE_valid_2000ex.txt --test cbtest_NE_test_2500ex.txt --output_dir test_output_cbt --evaluate_every_n 1e --patience_metric accuracy --own_eval --append_metaparams --no_html --dataset_type cbt --save_path test_output_cbt/model.blocks.pkl
+++ tee test_output_cbt/cbtest_NE_train.txt.-ehd_128_-sed_256.log -i
Using cuDNN version 5110 on context None
Mapped name None to device cuda: TITAN X (Pascal) (0000:22:00.0)
Traceback (most recent call last):
  File "text_comprehension/as_reader.py", line 166, in <module>
Output will be stored in test_output_cbt
Computing new vocabulary for file ../data/CBTest/data/cbtest_NE_train.txt.
Processed line 100000
Processed line 200000
Processed line 300000
Processed line 400000
Processed line 500000
Processed line 600000
Processed line 700000
Processed line 800000
Processed line 900000
Processed line 1000000
Processed line 1100000
Processed line 1200000
Processed line 1300000
Processed line 1400000
Processed line 1500000
Processed line 1600000
Processed line 1700000
Processed line 1800000
Processed line 1900000
Processed line 2000000
Processed line 2100000
Processed line 2200000
Processed line 2300000
STATISTICS
Total words: 53825761
Total distinct words: 60278
STATISTICS
Total words: 942535
Total distinct words: 10720


Added 1311 new words from file ../data/CBTest/data/cbtest_NE_test_2500ex.txt to previous vocabulary.
    exp.execute()
  File "/home//github/asreader/asreader/text_comprehension/text_comprehension_base.py", line 386, in execute
    **self.args.evaluate_every_n)
  File "/home//github/asreader/asreader/text_comprehension/monitoring.py", line 79, in __init__
    self._evaluator = MemoryDatasetEvaluator([context_attention, context,y_hat,y,candidates, candidates_mask, context_mask,x_mask,x], updates)
  File "/home//github/asreader/asreader/text_comprehension/monitoring.py", line 237, in __init__
    self._compile()
  File "/home//github/asreader/asreader/text_comprehension/monitoring.py", line 277, in _compile
    outputs)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/compile/function.py", line 326, in function
    output_keys=output_keys)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
    output_keys=output_keys)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/compile/function_module.py", line 1794, in orig_function
    output_keys=output_keys).create(
  File "/home//anaconda2/lib/python2.7/site-packages/theano/compile/function_module.py", line 1474, in __init__
    optimizer_profile = optimizer(fgraph)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 98, in __call__
    return self.optimize(fgraph)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 87, in optimize
    ret = self.apply(fgraph, *args, **kwargs)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 235, in apply
    sub_prof = optimizer.optimize(fgraph)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 87, in optimize
    ret = self.apply(fgraph, *args, **kwargs)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 235, in apply
    sub_prof = optimizer.optimize(fgraph)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/opt.py", line 87, in optimize
    ret = self.apply(fgraph, *args, **kwargs)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gpuarray/opt.py", line 382, in apply
    node.outputs)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gpuarray/opt.py", line 1189, in local_gpua_gemmbatch
    return gpugemmbatch_no_inplace(c, 1.0, a, b, 0.0)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gof/op.py", line 615, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/home//anaconda2/lib/python2.7/site-packages/theano/gpuarray/blas.py", line 391, in make_node
    assert A.ndim == 3
AssertionError

rkadlec · 2017-05-18T19:06:41Z

I have never seen something like that. Do you use the version of Blocks mentioned in readme.md? We also used older version of Theano than 0.9.0 as indicated in our logs. Let me know whether it helped.

mrcocytus · 2017-05-19T01:34:30Z

The same issue as above. The dependencies are quite annoying.
I guess it can be helpful that the author gives the whole pip list.

Seayoung277 · 2017-05-26T05:24:08Z

Got exactly the same problems and finally solved by:

Find the same commit version of Blocks. Actually I looked through all the commit sshs to find the desired one and finally downloaded the zip for installation.
Install Theano 0.8.0 instead of 0.9.0 before installing Blocks. If you install the old version Blocks first without Theano, it will install the latest verson Theano as well as numpy-1.9.3. You also have to reinstall numpy in order to correctly run the asreader. Older versions such as 1.9.3 or latest versions such as 1.12.1 can lead to different problems. I tried numpy-1.10.0 and it worked. I haven't tried other versions yet.

Currently the code is still running on my computer. I don't have a GPU so it'll take a long time. Till now everything seems alright.

byplusone · 2017-06-06T03:44:55Z

@seayoungzhang Did your code run successfully? I tried but still meet this problem. Can you show me your pip list? Thanks!

Seayoung277 · 2017-06-06T05:36:05Z

ALL PROBLEMS SOLVED

After following the steps presented above you may end in the error issued in #12
Please refer to the solution there.

Seayoung277 · 2017-06-06T06:03:27Z

And here's my pip list:

adium-theme-ubuntu (0.3.4)
alabaster (0.7.10)
Babel (2.4.0)
bleach (1.5.0)
blocks (0.2.0, /home/seayoung/blocks)
docutils (0.13.1)
fuel (0.2.0)
funcsigs (1.0.2)
h5py (2.7.0)
html5lib (0.9999999)
imagesize (0.7.1)
Jinja2 (2.9.6)
Markdown (2.2.0)
MarkupSafe (1.0)
mock (2.0.0)
nltk (3.2.4)
nose (1.3.7)
nose2 (0.6.5)
numexpr (2.6.2)
numpy (1.10.0)
olefile (0.44)
pbr (3.0.1)
picklable-itertools (0.1.1)
Pillow (4.1.1)
pip (9.0.1)
progressbar2 (3.6.0)
protobuf (3.3.0)
Pygments (2.2.0)
python-apt (1.1.0b1)
pytz (2017.2)
PyYAML (3.11)
pyzmq (16.0.2)
requests (2.14.2)
scipy (0.19.0)
setuptools (20.7.0)
six (1.10.0)
snowballstemmer (1.2.1)
Sphinx (1.6.1)
sphinx-rtd-theme (0.2.4)
sphinxcontrib-websupport (1.0.1)
tables (3.4.2)
tensorflow (1.2.0rc0)
Theano (0.8.0, /usr/local/lib/python2.7/dist-packages)
toolz (0.7.2)
tqdm (4.11.2)
typing (3.6.1)
unity-lens-photos (1.0)
Werkzeug (0.12.2)
wheel (0.29.0)
yadlt (0.0.6)

Some of them are irrelevant to this code. Just for reference.

purn3ndu · 2017-10-27T02:52:50Z

@seayoungzhang Do you have the commit ssh you used for installing blocks as you mentioned above? Thanks! :)

Seayoung277 · 2017-10-27T04:17:08Z

@purnendu91 359afad119f8c6ac0ebc3cc6ec6e6475656babae
It's provided in the project README file
Good Luck ! :)

fiberleif · 2017-11-28T11:14:21Z

step 1:
adding floatX: float32 in ~/.fuelrc
step 2:
~/.theanorc
[global]
device = cuda
floatX = float32

it seems work for my task @Actor-critic for Sequence Prediction

DAHANEYassine · 2018-06-02T18:34:49Z

someone help me please
ConfigurationError: Configuration not set and no default provided: floatX.
on windows 10, anaconda3

fmartinssp · 2018-06-23T20:39:12Z

DAHANEYassine Did you fix the problem? I have the same.

DAHANEYassine · 2018-07-17T11:26:34Z

fmartnissp i don't figure out how i can solve it.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX. #14

fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX. #14

wenwei202 commented May 17, 2017 •

edited

wenwei202 commented May 17, 2017

wenwei202 commented May 17, 2017

rkadlec commented May 18, 2017

mrcocytus commented May 19, 2017

Seayoung277 commented May 26, 2017 •

edited

byplusone commented Jun 6, 2017

Seayoung277 commented Jun 6, 2017

Seayoung277 commented Jun 6, 2017

purn3ndu commented Oct 27, 2017

Seayoung277 commented Oct 27, 2017

fiberleif commented Nov 28, 2017 •

edited

DAHANEYassine commented Jun 2, 2018

fmartinssp commented Jun 23, 2018

DAHANEYassine commented Jul 17, 2018

fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX. #14

fuel.exceptions.ConfigurationError: Configuration not set and no default provided: floatX. #14

Comments

wenwei202 commented May 17, 2017 • edited

wenwei202 commented May 17, 2017

wenwei202 commented May 17, 2017

rkadlec commented May 18, 2017

mrcocytus commented May 19, 2017

Seayoung277 commented May 26, 2017 • edited

byplusone commented Jun 6, 2017

Seayoung277 commented Jun 6, 2017

Seayoung277 commented Jun 6, 2017

purn3ndu commented Oct 27, 2017

Seayoung277 commented Oct 27, 2017

fiberleif commented Nov 28, 2017 • edited

DAHANEYassine commented Jun 2, 2018

fmartinssp commented Jun 23, 2018

DAHANEYassine commented Jul 17, 2018

wenwei202 commented May 17, 2017 •

edited

Seayoung277 commented May 26, 2017 •

edited

fiberleif commented Nov 28, 2017 •

edited