Add pop2piano

[ArthurZucker]

Model description

• Introduce a large amount of paired and synchronised {pop, piano cover} data using an automated pipeline.
• Pop2Piano, a Transformer network that generates piano covers given waveforms of pop music.
• First model to directly generate a piano cover from pop audio without melody and chord extraction modules.
• Uses a T5 model so should be straightforward.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

• Weights : https://github.com/sweetcocoa/pop2piano/releases/download/dpi_2k_epoch/model-1999-val_0.67311615.ckpt
• Code : https://github.com/sweetcocoa/pop2piano/
• Paper : https://arxiv.org/abs/2211.00895
• Colab : https://colab.research.google.com/drive/1rBAs2TkryDnnQOhcM-mtlrgtL2h3ekml?usp=sharing

[susnato]

Hi @ArthurZucker could you please share the progress of the model addition?(I am asking because the last time this Issue has had any actions was in Nov 8, 2022). I tried this model on colab and really loved it. I want to add / help adding this model to HF. Is it possible that we can collaborate in this addition?

[ArthurZucker]

Hey! I did not start at all! Feel free to open a PR and ping me for pointers/help! I won't have time to do it alone but would love to collaborate ! 😉

[susnato]

@ArthurZucker Ok, my term exams are till 3rd March so I will start working from 4th, in meantime I will open a PR. Do you want to continue communicating through that PR or how about communicating through slack/discord if it's possible?

[ArthurZucker]

Sure, will invite you to slack if you can share your email, arthur@hf.co!

[susnato]

Sure, will invite you to slack if you can share your email, arthur@hf.co!

My email is - susnatodhar10@gmail.com

@ArthurZucker

[ArthurZucker]

Just invited you! Good luck on your mid terms 😉

[ArthurZucker]

Also if anyone want to tackle this before, ping me and will add you to the channel

[manhdoan291]

@ArthurZucker How to retrain model and how to get datasets to retrain? ! Help me