NLP isn't getting imported due to ApexGuardDefaults

[adi7820]

Describe the bug

I've installed Nemo using below commands in my aws sagemaker

!sudo yum update -y && yum install -y libsndfile1 ffmpeg
!pip install Cython packaging
BRANCH='main'
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@{BRANCH}#egg=nemo_toolkit[all]

but getting error while I was trying anything related to NLP

from nemo.collections import nlp

Error:

ModuleNotFoundError Traceback (most recent call last)
File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/models/language_modeling/megatron/gpt_layer_modelopt_spec.py:16
15 try:
---> 16 from megatron.core.fusions.fused_bias_dropout import get_bias_dropout_add
17 from megatron.core.tensor_parallel.layers import ColumnParallelLinear, RowParallelLinear

ModuleNotFoundError: No module named 'megatron'

During handling of the above exception, another exception occurred:

NameError Traceback (most recent call last)
Cell In[8], line 1
----> 1 import nemo.collections.nlp

File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/init.py:15
1 # Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 15 from nemo.collections.nlp import data, losses, models, modules
16 from nemo.package_info import version
18 # Set collection version equal to NeMo version.

File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/models/init.py:28
23 from nemo.collections.nlp.models.information_retrieval import BertDPRModel, BertJointIRModel
24 from nemo.collections.nlp.models.intent_slot_classification import (
25 IntentSlotClassificationModel,
26 MultiLabelIntentSlotClassificationModel,
27 )
---> 28 from nemo.collections.nlp.models.language_modeling import MegatronGPTPromptLearningModel
29 from nemo.collections.nlp.models.language_modeling.bert_lm_model import BERTLMModel
30 from nemo.collections.nlp.models.language_modeling.transformer_lm_model import TransformerLMModel

File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/models/language_modeling/init.py:16
1 # Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
15 from nemo.collections.nlp.models.language_modeling.bert_lm_model import BERTLMModel
---> 16 from nemo.collections.nlp.models.language_modeling.megatron_gpt_prompt_learning_model import (
17 MegatronGPTPromptLearningModel,
18 )
19 from nemo.collections.nlp.models.language_modeling.megatron_retrieval_model import MegatronRetrievalModel
20 from nemo.collections.nlp.models.language_modeling.megatron_retro_model import MegatronRetroModel

File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/models/language_modeling/megatron_gpt_prompt_learning_model.py:31
27 from nemo.collections.nlp.metrics.prompt_learning_metrics import AccuracyScore, BLEUScore, ROUGEScores
28 from nemo.collections.nlp.models.language_modeling.megatron_base_prompt_learning_model import (
29 MegatronBasePromptLearningModel,
30 )
---> 31 from nemo.collections.nlp.models.language_modeling.megatron_gpt_model import MegatronGPTModel
32 from nemo.collections.nlp.modules.common import VirtualPromptPlaceholderToken, VirtualPromptSource, VirtualPromptStyle
33 from nemo.collections.nlp.modules.common.megatron.utils import (
34 ApexGuardDefaults,
35 average_losses_across_data_parallel_group,
36 get_iterator_k_split,
37 )

File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py:44
40 from nemo.collections.nlp.models.language_modeling.megatron.falcon.falcon_spec import get_falcon_layer_spec
41 from nemo.collections.nlp.models.language_modeling.megatron.gpt_full_te_layer_autocast_spec import (
42 get_gpt_full_te_layer_autocast_spec,
43 )
---> 44 from nemo.collections.nlp.models.language_modeling.megatron.gpt_layer_modelopt_spec import get_gpt_layer_modelopt_spec
45 from nemo.collections.nlp.models.language_modeling.megatron.gpt_model import GPTModel
46 from nemo.collections.nlp.models.language_modeling.megatron_base_model import MegatronBaseModel

File ~/SageMaker/nemo_nlp/NeMo/nemo/collections/nlp/models/language_modeling/megatron/gpt_layer_modelopt_spec.py:31
27 HAVE_MEGATRON_CORE = True
29 except (ImportError, ModuleNotFoundError) as e:
---> 31 TransformerLayer = TransformerLayerSubmodules = ApexGuardDefaults
32 MLP = MLPSubmodules = ModuleSpec = IdentityOp = ApexGuardDefaults
33 AttnMaskType = DotProductAttention = TENorm = ApexGuardDefaults

NameError: name 'ApexGuardDefaults' is not defined

[titu1994]

Nlp domain requires a lot of external dependencies such as Apex, Transformer Engine and Megatron Core which are not easily pip installable. Please use the container instructions for them, or follow instructions on the readme to build those libraries for a local installation

[ashors1]

Hi @adi7820, the issue you are seeing should be resolved by this PR. Are you able to verify that this is the case with your setup?