Remove deprecated arguments from TE's TransformerLayer

[jbaczek]

What does this PR do ?

Transformer engine doesn't use this arguments since v0.7 and now in v1.0 they removed them completely.

Collection: [Note which collection this PR will affect]
NLP

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• [x ] Bugfix
• Documentation

Who can review?

@ericharper

[aklife97]

the PR changes look fine but we might want to note that this only changes things for NeMo local GPT models, not m-core based ones which are the ones we majorly use now

[ericharper]

I think the main issue is that user are used to configuring this from the config. Now TE expects an environment variable to be set. Mcore makes sure that the env var and the config are consistent. Can NeMo set the env var automatically so that the user doesn't have to set the env var?

[aklife97]

yes, thats right. It would probably be better if we automatically set the TE env variable if someone passes it in the config

[aklife97]

Ideally, I would prefer if this env var setting happens in m-core since m-core also relies on arg passing format and is the place where we actually interface with TE and setup TE/local layers. But if we feel that it might need broader discussion it might make sense to add this in nemo for now so that it unblocks the issue?

[ShriyaRishab]

@ericharper @aklife97 can you approve this for now in nemo so it unblocks?

[ericharper]

LGTM. Thanks!

[ericharper]

jenkins

[ShriyaRishab]

jenkins