restore `register_external_parameter` doc

[stas00]

Clearly we still need register_external_parameter, so we probably should restore:
https://www.deepspeed.ai/tutorials/zero/#registering-external-parameters
to 0.3.15-state.

[lintangsutawika]

I'm currently exploring deepspeed and also have a usecase where I have to access the weights of the model embeddings during training.

This feature is something I need

[tjruwase]

@stas00, @lintangsutawika, thanks for reporting this issue. This is really a problem for confusing documentation as the feature is still there but not deprecated as suggested in the tutorial below.

The text was intended to communicate that manual parameter registration may not be needed because of the automatic parameter registration support. We will fix the tutorial text to avoid this confusion. In the meantime, please our readthedocs on general manual parameter coordination and the more specific registering external parameters. Please let us know of any issues you encounter. Thanks!

[lintangsutawika]

I'm still trying to understand how to use this.

My usecase is exposing the embeddings weight and multiplying in manually. This is because the sampled_input are differentiable one hot encodings.

sampled_input_embed = sampled_input.view(b*l,c).mm(model.module.shared.weight)

So to manually registered the shared.weight I implemented a wrapper on the class I want to use.

    class Model(AutoModelForSeq2SeqLM):
        def __init__(self, *args):
            super().__init__(self, *args)

            deepspeed.zero.register_external_parameter(self, self.shared.weight)

and initiate as usual

    model = Model.from_pretrained(
        model_args.model_name_or_path,
        from_tf=".ckpt" in model_args.model_name_or_path,
        config=config,
        cache_dir=model_args.cache_dir,
    )

But this doesn't seem to work as well.

File "main.py", line 145, in compute_loss
    sampled_input_embed = sampled_input.view(b*l,c).mm(model.module.shared.weight)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)