Regarding Prefix Tuning citation

[zphang]

Genuinely curious: For Prefix Tuning, what's the reason for citing P-Tuning v2 rather than the Prefix Tuning paper?

[pacman100]

Please find the reasoning below:

1. P-Tuning v2 already cites Prefix Tuning

2. P-Tuning v2 uses classification heads and not the verbalizers for sequence labelling tasks. We find that this results in great performance. Example: prefix_tuning.ipynb

3. Our code is based on P-Tuning V2 repo and clearly mentioned here:
   https://github.com/huggingface/peft/blob/main/src/peft/tuners/prefix_tuning.py#L47-L49

Feel free to raise a PR to add citation to Prefix Tuning paper in README

[zphang]

1. I don't think that's a good reason for citing P-Tuning v2 as the primary citation - that would be the equivalent of citing RoBERTa instead of BERT when describing BERT.
2. Prefix Tuning (Li and Liang, 2021) doesn't use verbalizers either. Specifically, Prefix Tuning was introduced for NLG applications, and P-Tuning v2 adapted it for NLU applications. Given that the repository also supports generation models, and that the Prefix Tuning paper came out first, it seems odd to not cite it and only cite P-Tuning v2. Insofar as P-Tuning v2 introduced combining Prefix Tuning with classification heads, I can see that being an argument for it being cited, but only secondarily to the Li and Liang paper which first proposed Prefix Tuning. The P-Tuning v2 paper itself mentions that "P-Tuning v2 is an implementation of [Prefix Tuning] optimized and adapted for NLU".
3. Likewise, the source of implementation doesn't seem like a good reason for the choice of citation. Credit for code should be given to the P-Tuning v2 folks, but one would not cite the HF Transformers paper when they are referring to a specific LM model.

(Apologies if this sounds like nitpicking, but I am particular about this because there has been a lot of confusion over time around the naming and lineage of PEFT methods, and especially for a high-visibility repository like HF/PEFT, I think it helps to be precise.)

I think the ideal citation format for Prefix Tuning should be the Prefix Tuning paper, and then secondarily the P-Tuning v2 paper. I can submit a PR for this.

[pacman100]

Feel free to raise a PR to add citation to Prefix Tuning paper in README

Hello, makes sense. As mentioned, please go ahead and submit the PR for this.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.