Prefix_Adapter& MHAdapter_Layer in the petl/petl_factory.py seems to be not used

[Doragd]

"Adapter" is not defined.

class Prefix_Adapter(nn.Module):
    def __init__(self, args, config):
        super().__init__()
        self.prefix = Prefix(args, config)
        self.adapters = Adapter(args, config) # "Adapter" is not defined 

    def forward(self, bsz, nsamples=1, device="cuda"):
        prefix = self.prefix(bsz, nsamples, device)
        adapters = self.adapters(bsz)

        for ii, dic in enumerate(adapters):
            for key, value in dic.items():
                prefix[ii][key] = value
        return prefix

[Doragd]

• embed_dim is not defined: https://github.com/jxhe/unify-parameter-efficient-tuning/blob/master/src/transformers/models/roberta/modeling_roberta.py#L203
• src_len is not defined: https://github.com/jxhe/unify-parameter-efficient-tuning/blob/master/src/transformers/models/roberta/modeling_roberta.py#L349
• src_len is not defined: https://github.com/jxhe/unify-parameter-efficient-tuning/blob/master/src/transformers/models/roberta/modeling_roberta.py#L350

[jxhe]

Thank you for capturing these oversights! We just fixed them