Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-speech sounds like suno bark ? #61

Open
Revln9 opened this issue Apr 3, 2024 · 4 comments
Open

non-speech sounds like suno bark ? #61

Revln9 opened this issue Apr 3, 2024 · 4 comments

Comments

@Revln9
Copy link

Revln9 commented Apr 3, 2024

is it possible to generate interjections with this model like mhmm or aha ?

@jasonppy
Copy link
Owner

jasonppy commented Apr 4, 2024

Thanks!
There are two possible way to have interjections:

  1. If the prompt has sounds like that, it's possible that the model will add that to the generation spontaneously
  2. if you wrote these in the target transcript, the model will just follow

@yoesak
Copy link
Contributor

yoesak commented Apr 20, 2024

Thanks! There are two possible way to have interjections:

  1. If the prompt has sounds like that, it's possible that the model will add that to the generation spontaneously
  2. if you wrote these in the target transcript, the model will just follow

So theoretically, VoiceCraft can be used to generate singing melody (vocal only) ?

@jasonppy
Copy link
Owner

Thanks! There are two possible way to have interjections:

  1. If the prompt has sounds like that, it's possible that the model will add that to the generation spontaneously
  2. if you wrote these in the target transcript, the model will just follow

So theoretically, VoiceCraft can be used to generate singing melody (vocal only) ?

I think it's unlikely to generate singing voice out-of-the-box, just because the training data is speech data. You could finetune it on singing data

@yoesak
Copy link
Contributor

yoesak commented Apr 21, 2024

Cool, I will try with singing data. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants