fix retie_parameters

[SunMarc]

What does this PR do ?

This PR fixes the retie_parameters function. The issue is that we were making the assumption that the first elem of the tied_params was the param to tie. This should solves the BNB tests that are failing on the CI because we switched to safetensors.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[muellerzr]

Thanks!

[BenjaminBossan]

Looks good, thanks.

A question: Something that can happen now is that no parameters are being tied, right? (when param_to_tie is None after finishing the first loop.) Can this be problematic?

Also, just a tip, code like this can be simplified with the attrgetter function from the operator module. E.g. in the first loop:

        for param_name in tied_group:
            param = attrgetter(param_name)(module)
            if param_to_tie is ...

should work.

In the second loop, a bit of extra work is required:

            for param_name in tied_group:
                mod, _, param = param_name.rpartition(".")
                module = attrgetter(mod)(model)
                setattr(module, param, param_to_tie)

No need to change here, just something I find neat.

[SunMarc]

A question: Something that can happen now is that no parameters are being tied, right? (when param_to_tie is None after finishing the first loop.) Can this be problematic?

If param_to_tie is None, it means that all the tensors are on the meta device and they should also have the same shape/dtype since they were tied before (here it is a retie function). So I think it should be fine.

Also, just a tip, code like this can be simplified with the attrgetter function from the operator module. E.g. in the first loop:

Indeed! Thx for sharing, I will use that more often from now =)