Latest huggingface transformers version breaking nlp modules

[sanjams2]

Describe the bug

The latest version of transformers (4.41.0) breaks MegatronGPTModel. That is because of this commit which removes the ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST constant which is used by the huggingface_utils.py module here. This huggingface_utils.py is an indirect dependency of nemo/collections/nlp.

Steps/Code to reproduce bug

git clone https://github.com/NVIDIA/NeMo.git
cd Nemo
python3 -m venv .
source bin/activate
pip install -r requirements/requirements_common.txt
pip install -r requirements/requirements_lightning.txt
pip install -r requirements/requirements_nlp.txt
python -c 'from nemo.collections.nlp.models.language_modeling.megatron_gpt_model import MegatronGPTModel'

Output:

[NeMo W 2024-05-22 04:24:07 nemo_logging:393] /NeMo/nemo/collections/nlp/modules/common/bert_module.py:57: SyntaxWarning: invalid escape sequence '\.'
      m = re.match("^bert.*?\.", list(pretrained_dict.keys())[0])

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/NeMo/nemo/collections/nlp/__init__.py", line 15, in <module>
    from nemo.collections.nlp import data, losses, models, modules
  File "/NeMo/nemo/collections/nlp/data/__init__.py", line 42, in <module>
    from nemo.collections.nlp.data.zero_shot_intent_recognition.zero_shot_intent_dataset import (
  File "/NeMo/nemo/collections/nlp/data/zero_shot_intent_recognition/__init__.py", line 16, in <module>
    from nemo.collections.nlp.data.zero_shot_intent_recognition.zero_shot_intent_dataset import (
  File "/NeMo/nemo/collections/nlp/data/zero_shot_intent_recognition/zero_shot_intent_dataset.py", line 30, in <module>
    from nemo.collections.nlp.parts.utils_funcs import tensor2list
  File "/NeMo/nemo/collections/nlp/parts/__init__.py", line 17, in <module>
    from nemo.collections.nlp.parts.utils_funcs import list2str, tensor2list
  File "/NeMo/nemo/collections/nlp/parts/utils_funcs.py", line 37, in <module>
    from nemo.collections.nlp.modules.common.megatron.utils import erf_gelu
  File "/NeMo/nemo/collections/nlp/modules/__init__.py", line 16, in <module>
    from nemo.collections.nlp.modules.common import (
  File "/NeMo/nemo/collections/nlp/modules/common/__init__.py", line 18, in <module>
    from nemo.collections.nlp.modules.common.huggingface import (
  File "/NeMo/nemo/collections/nlp/modules/common/huggingface/__init__.py", line 19, in <module>
    from nemo.collections.nlp.modules.common.huggingface.huggingface_utils import (
  File "/NeMo/nemo/collections/nlp/modules/common/huggingface/huggingface_utils.py", line 18, in <module>
    from transformers import (
ImportError: cannot import name 'ALBERT_PRETRAINED_MODEL_ARCHIVE_LIST' from 'transformers' (/NeMo/lib/python3.12/site-packages/transformers/__init__.py)

Expected behavior

Importing MegatronGPTModel (or other classes from these modules) should not fail.

Environment details

• Environment location: Cloud AWS
• Method of NeMo install: from source
• OS version: 5.10.216-182.855.amzn2int.x86_64
• PyTorch version: 2.3.0+cu121
• Python version: Python 3.12.3

[titu1994]

Thanks for informing us of this ! Pr #9261 was just merged into 2.0 RC 1 branch.

For main, we will try to either patch it or also pin HF temporarily while we cleanup the NLP domain

[sanjams2]

Thanks @titu1994. Should the issue not stay open until main gets fixed though?

[ericharper]

pr to main is here: #9273, we'll merge once ci passes

[titu1994]

Oh it auto closed due to pr link, opening it again

[ericharper]

PR to main is merged.

[FabrizioRicciarelli]

Still broken...

[adamlin120]

still broken

[mbugert]

Works for me with pip install transformers==4.40.0.

[chetan-atmecs]

@mbugert thanks bro it works for me with. bro can you give me some resources about rag implementation with nemo. we are not able to find