datasets.load_metric() function is not working?

[Autumn1994]

Hi,

I'm trying to follow the tutorial on text classification, however, when I call load_metrics(), it throws out the following error message:

AttributeError Traceback (most recent call last)
in
1 actual_task = "mnli" if task == "mnli-mm" else task
----> 2 metric = load_metric('glue', actual_task)
3 metric
~/.local/lib/python3.6/site-packages/datasets/load.py in load_metric(path, config_name, process_id, num_process, cache_dir, experiment_id, keep_in_memory, download_config, download_mode, script_version, **metric_init_kwargs)
498 dataset=False,
499 )
--> 500 metric_cls = import_main_class(module_path, dataset=False)
501 metric = metric_cls(
502 config_name=config_name,
~/.local/lib/python3.6/site-packages/datasets/load.py in import_main_class(module_path, dataset)
64 """
65 importlib.invalidate_caches()
---> 66 module = importlib.import_module(module_path)
67
68 if dataset:
/usr/lib/python3.6/importlib/init.py in import_module(name, package)
124 break
125 level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)
127
128
/usr/lib/python3.6/importlib/_bootstrap.py in _gcd_import(name, package, level)
/usr/lib/python3.6/importlib/_bootstrap.py in find_and_load(name, import)
/usr/lib/python3.6/importlib/_bootstrap.py in find_and_load_unlocked(name, import)
/usr/lib/python3.6/importlib/_bootstrap.py in _load_unlocked(spec)
/usr/lib/python3.6/importlib/_bootstrap_external.py in exec_module(self, module)
/usr/lib/python3.6/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)
~/.cache/huggingface/modules/datasets_modules/metrics/glue/e4606ab9804a36bcd5a9cebb2cb65bb14b6ac78ee9e6d5981fa679a495dd55de/glue.py in
103
104
--> 105 @datasets.utils.file_utils.add_start_docstrings(_DESCRIPTION, _KWARGS_DESCRIPTION)
106 class Glue(datasets.Metric):
107 def _info(self):
AttributeError: module 'datasets.utils.file_utils' has no attribute 'add_start_docstrings'

I was able to successfully reproduce the fine-tuning process a month ago but got the error above today. The code are completely the same as the notebook. Any ideas on what might go wrong? Thanks a lot!

[Autumn1994]

Got it fixed. Thanks!

[cmcmaster1]

Got it fixed. Thanks!

I don't like re-opening this, but could you share some insight into what was the problem and how you fixed it? I'm having the same issue.

[satriowputra]

Got it fixed. Thanks!

I don't like re-opening this, but could you share some insight into what was the problem and how you fixed it? I'm having the same issue.

Make sure you are using the latest version of datasets. Run pip install --upgrade datasets to upgrade your datasets library.

[nsaphra]

Hi, can we reopen this issue? I'm having the same problem on datasets 1.2.1 (which is the current version in conda).

>>> from datasets import load_metric
>>> metric = load_metric("glue", "mnli")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ext3/miniconda3/lib/python3.8/site-packages/datasets-1.2.1-py3.8.egg/datasets/load.py", line 502, in load_metric
  File "/ext3/miniconda3/lib/python3.8/site-packages/datasets-1.2.1-py3.8.egg/datasets/load.py", line 66, in import_main_class
  File "/ext3/miniconda3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ns4008/.cache/huggingface/modules/datasets_modules/metrics/glue/e4606ab9804a36bcd5a9cebb2cb65bb14b6ac78ee9e6d5981fa679a495dd55de/glue.py", line 105, in <module>
    @datasets.utils.file_utils.add_start_docstrings(_DESCRIPTION, _KWARGS_DESCRIPTION)
AttributeError: module 'datasets.utils.file_utils' has no attribute 'add_start_docstrings'

[Autumn1994]

Sorry for the late reply. The error message is more or less confusing. My case/experience was that the error was nothing related to this function. Check and make sure your packages/environment/dependencies are updated and compatible, including pip and python wheels etc. Please let me know if you experience any other questions. Thx.

[nsaphra]

@Autumn1994 What version are you using of datasets and python?

[Autumn1994]

Python 3.6.9 and 1.2.1 for datasets.

[nsaphra]

I am using 1.2.1 as well. Could you give some more details of how you fixed your problem? It seems that we had the same error thrown by the same function.

[Autumn1994]

@nsaphra I couldn't remember every steps I took as I didn't have a clear path that time and it's a bit self-exploratory. Sorry about that. My guess is that the error was caused by the recent updates of python wheels (Dec 2020). Check that first and then reinstall the packages (not a must) and see if that works.

[imbesat-rizvi]

@nsaphra I just faced the same issue. I was using 1.2.1 from conda and received the same AttributeError complaining about 'add_start_docstrings'. Uninstalling the conda installed datasets and then installing the latest datasets (version 1.5.0) using pip install solved the issue for me. I don't like mixing up conda and pip installs in the same environments but this will have to do for now, until 1.5.0 is made available through conda.