The library dose not work with the latest allennlp

[apohllo]

Contrary to the instructions, I installed the latest allennlp library. Unfortunately I got the following error:

Traceback (most recent call last):
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 918, in save_global
    obj2, parent = _getattribute(module, name)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 266, in _getattribute
    .format(name, obj))
AttributeError: Can't get local attribute 'wrap_function.<locals>.WrappedFunc' on <function wrap_function at 0x2b6ace9a7d90>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 639, in save_global
    return Pickler.save_global(self, obj, name=name)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 922, in save_global
    (obj, module_name, name))
_pickle.PicklingError: Can't pickle <class 'ray.tune.trainable.wrap_function.<locals>.WrappedFunc'>: it's not found as ray.tune.trainable.wrap_function.<locals>.WrappedFunc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/net/people/plgapohl/python-albert-pytorch/bin/allentune", line 11, in <module>
    load_entry_point('allentune', 'console_scripts', 'allentune')()
  File "/net/people/plgapohl/allentune/allentune/commands/__init__.py", line 67, in main
    args.func(args)
  File "/net/people/plgapohl/allentune/allentune/commands/search.py", line 126, in search_from_args
    executor.run(args)
  File "/net/people/plgapohl/allentune/allentune/modules/ray_executor.py", line 94, in run
    self.run_distributed(run_func, args)
  File "/net/people/plgapohl/allentune/allentune/modules/ray_executor.py", line 58, in run_distributed
    register_trainable("run", run_func)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/tune/registry.py", line 49, in register_trainable
    _global_registry.register(TRAINABLE_CLASS, name, trainable)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/tune/registry.py", line 88, in register
    self._to_flush[(category, key)] = pickle.dumps(value)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 881, in dumps
    cp.dump(obj)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 268, in dump
    return Pickler.dump(self, obj)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 648, in save_global
    return self.save_dynamic_class(obj)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 495, in save_dynamic_class
    save(clsdict)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
   save(v)                                                                                                                                                                                        [26/1833]
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 410, in save_function
    self.save_function_tuple(obj)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 553, in save_function_tuple
    save(state)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 781, in save_list
    self._batch_appends(obj)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 808, in _batch_appends
    save(tmp[0])
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 410, in save_function
    self.save_function_tuple(obj)
  File "/net/people/plgapohl/python-albert-pytorch/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py", line 553, in save_function_tuple
    save(state)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
  f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 781, in save_list
    self._batch_appends(obj)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 808, in _batch_appends
    save(tmp[0])
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/net/software/local/python/3.6.5/lib/python3.6/pickle.py", line 496, in save
    rv = reduce(self.proto)
TypeError: can't pickle _thread.RLock objects

Any idea how I can fix that?

[apohllo]

To my complete confusion, this is the last version that still works:

• allenai/allennlp@11f4307
  and this is the first version that doesn't:
• allenai/allennlp@26e313b

The only difference is the bump in version from rc3 to rc4.

[apohllo]

I changed the testing procedure (I installing from local git repot now) and found out, that the 26e313b version still, works, but the next version doesn't:
allenai/allennlp@4a6023b

I will investigate the differences between these version.

[apohllo]

Well, reverting that commit (4a6023b) makes the more recent version (592c65341891ccadec31a624c4e6f062e8455723 from 13th may) of allennlp work properly with allentune. It seems that the problem lays in that commit. Still, I have no idea why changing the logging infrastructure causes the error.

[ofersabo]

What latest version of allennlp is compatible with allentune? I'm talking about official releases of allennlp, i.e. not a source installation.

[apohllo]

@ofersabo I haven't tried it but I suppose it should work with either v0.8.5 or v0.9.0 since these are the commits closest to the commit mentioned in the documentation.

[ofersabo]

@apohllo on my end the installation isn't functional even in 0.9.0 and. 0.8.5 versions.

I get these error messages:
============================= test session starts ==============================
platform linux -- Python 3.6.10, pytest-5.4.3, py-1.9.0, pluggy-0.13.1 -- /home/nlp//anaconda3/envs/tune/bin/python
cachedir: .pytest_cache
rootdir: /home/nlp//translation/mt_ex_2/allentune, inifile: pytest.ini
plugins: flaky-3.6.1, cov-2.10.0
collected 4 items / 1 error / 3 selected

==================================== ERRORS ====================================
__________________ ERROR collecting tests/test_example_run.py __________________
ImportError while importing test module '/home/nlp//translation/mt_ex_2/allentune/tests/test_example_run.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_example_run.py:1: in
from allentune.modules import AllenNlpRunner, RayExecutor
allentune/modules/init.py:1: in
from allentune.modules.allennlp_runner import AllenNlpRunner
allentune/modules/allennlp_runner.py:14: in
from allennlp.common.util import import_module_and_submodules
E ImportError: cannot import name 'import_module_and_submodules'
=============================== warnings summary ===============================
/home/nlp//anaconda3/envs/tune/lib/python3.6/site-packages/plac_ext.py:6
/home/nlp/____/anaconda3/envs/tune/lib/python3.6/site-packages/plac_ext.py:6: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================== short test summary info ============================
ERROR tests/test_example_run.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 5.73s ==========================

When I'm trying to run a test directly I get this error:

from allennlp.common.util import import_module_and_submodules
ImportError: cannot import name 'import_module_and_submodules'

these errors are in allennlp==0.9.0

[apohllo]

I encountered the same error. As far as I remember, commenting out the line with the import fixed it.

[ofersabo]

OK thanks, so I have to install from source...

[kernelmachine]

Hi all, sorry about the delay on this. This should be fixed by #12. Feel free to re-open if you're still having issues -- make sure you use allennlp 1.0.