AttributeError: 'ORTModule' object has no attribute 'resize_token_embeddings'

[bmedishe]

Hi,
I am using ort to run transformers/examples/pytorch/language-modeling/run_clm.py (fine-tuning GPT-2 on WikiText-2, using the raw WikiText-2 no tokens were replaced before the tokenization). I am running it on rocm platform.
I edited the script like this

from torch_ort import ORTModule

    if model_args.model_name_or_path:
        model = AutoModelForCausalLM.from_pretrained(
            model_args.model_name_or_path,
            from_tf=bool(".ckpt" in model_args.model_name_or_path),
            config=config,
            cache_dir=model_args.cache_dir,
            revision=model_args.model_revision,
            use_auth_token=True if model_args.use_auth_token else None,
        )
        model = ORTModule(model)
    else:
        model = AutoModelForCausalLM.from_config(config)
        model = ORTModule(model)
        n_params = sum(dict((p.data_ptr(), p.numel()) for p in model.parameters()).values())
        logger.info(f"Training new model from scratch - Total size={n_params/2**20:.2f}M params")

I am getting this error

Traceback (most recent call last):
  File "./examples/pytorch/language-modeling/run_clm.py", line 519, in <module>
    main()
  File "./examples/pytorch/language-modeling/run_clm.py", line 353, in main
    model.resize_token_embeddings(len(tokenizer))
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 948, in __getattr__
    type(self).__name__, name))
AttributeError: 'ORTModule' object has no attribute 'resize_token_embeddings'

Could you kindly help me in resolving it
Thank you
Bhavya

[bmedishe]

replacing
model.resize_token_embeddings(len(tokenizer))
with

model_to_resize` = model.module if hasattr(model, 'module') else model
model_to_resize.resize_token_embeddings(len(tokenizer))

src : huggingface/transformers#7146

does not give the above error message , I would like to know if this is a right fix.

[suffiank]

Hi Bhavya,

In general ORTModule does not forward the attributes of the underlying model. For now, yes, this is the correct fix. However, this API is subject to change as exposing the attribute .module to get the underlying model has led to issues elsewhere. Likely the name will change to something bit less friendly, e.g. ._original_module.

For HF-GPT2, you should be able to use the following repository as-is:
https://github.com/microsoft/huggingface-transformers

In the above, the ORTModule is inserted in the huggingface trainer.py script itself:
https://github.com/microsoft/huggingface-transformers/blob/c1b959563ebb677f744382ea95ca891295092187/src/transformers/trainer.py#L1109

And there's another tweak here to ensure the DDP wrapping occurs correctly for >1 gpu:
https://github.com/microsoft/huggingface-transformers/blob/c1b959563ebb677f744382ea95ca891295092187/src/transformers/trainer.py#L926

I run the model using the following launch command:
python -m torch.distributed.launch --nproc_per_node 8 huggingface-transformers/examples/pytorch/language-modeling/run_clm.py --model_name_or_path gpt2 --dataset_name wikitext --dataset_config_name wikitext-2-raw-v1 --do_train --label_smoothing 0.1 --max_steps 260 --logging_steps 1 --overwrite_output_dir --output_dir gpt2-results --logging_dir gpt2-tensorboard --per_device_train_batch_size 8 --fp16 --dataloader_num_workers 1 --ort --skip_memory_metrics

Let me know if you have other issues.

-- Suffian

[bmedishe]

Thank you Suffian. I will try from https://github.com/microsoft/huggingface-transformers.
Bhavya

[natke]

Hi @bmedishe, is your issue resolved now?

[bmedishe]

Yes @natke Thank you

[natke]

Great, thanks. I will close this issue. Please reach out again if you need to.