Add text to audio task

[Vaibhavs10]

No description provided.

[osanseviero]

You will also need to update in a couple of other places https://github.com/search?q=repo%3Ahuggingface%2Fhub-docs+TEXT-to-speech+language%3ATypeScript&type=code&l=TypeScript

[Vaibhavs10]

I think I added it to all the relevant ones. Do you mind giving it a look again please @osanseviero ?

[osanseviero]

Thank you! Please check that you have similar set of files as in https://github.com/search?q=repo%3Ahuggingface%2Fhub-docs+TEXT-to-speech+language%3ATypeScript&type=code&l=TypeScript

• You also need to add it to tasksData (can lead to an undefined task page for now (or same as TTS?) - in any case, if not, let's open an issue for a follow-up task page
• Need to specify which libraries (and here) support this task

[osanseviero]

Awesome, things are looking good! You will also need to

• Add an icon (similar to https://github.com/huggingface/hub-docs/blob/main/js/src/lib/components/Icons/IconTextToSpeech.svelte ) and specify it in https://github.com/huggingface/hub-docs/blob/main/js/src/lib/components/PipelineIcon/PipelineIcon.svelte#L27 (I suggest to check https://carbondesignsystem.com/ for icons)
• You might also want to enable the widget, which involves modifying https://github.com/huggingface/hub-docs/blob/main/js/src/lib/components/InferenceWidget/InferenceWidget.svelte (most likely you can re-use TextToSpeechWidget)

You can preview how this would look like in GitHub Codespaces as specified in https://github.com/huggingface/moon-landing#codespace

[Vaibhavs10]

Noice. Going to to test this now!

[Vaibhavs10]

Weird! I keep getting permission denied on codespace for anything I do. Is this the same for you too @osanseviero ?

[Vaibhavs10]

Hey @mishig25

After your suggestions, I tried running npm ci and npm run prettier - no changes
then I tried npm run lint also resulted in no changes.

Send help! - welp!

[mishig25]

Was TextToAudio widget was created in this PR ?
if so, it would good idea to provide a description on what the API shape looks like, plus screenshot, plus which model to test on. You can see an example here

After your suggestions, I tried running npm ci and npm run prettier - no changes
then I tried npm run lint also resulted in no changes.

npm run format:all was the one

hub-docs/js/package.json

Line 14 in b1a591d

"format:all": "npm run prettier src && npm run lint:all",

[Vaibhavs10]

Hey @mishig25 - I just copied InferenceTextToSpeech widget over to text-to-audio.
So there are no new changes. I'll follow up with a separate PR to change the actual look.

Let me know if this is okay for you?

[osanseviero]

I think we should reuse TextToSpeechWidget rather than reimplementing it, same way we reuse TabularDataWidget. API input/output is exactly the same

[mishig25]

Btw, is the task already supported in api-inference? Is there a model or curl script to test on ?

[Vaibhavs10]

Hey! @mishig25 - Sorry for the delayed response, api-inference needed an update to make sure that the model can work.

Since the pipeline_tag text-to-audio is not yet supported the models cannot be directly curled, however, changing the pipeline_tag to text-to-speech does the trick and works - you can test it here: https://huggingface.co/reach-vb/musicgen-small?text=lo-fi+music

Do note that this is embarrassingly slow - will work on fixing that separately.

@osanseviero - I was thinking of making a follow-up PR to update the Icon for TTM, that's why I created those files. I think it'd be good to distinguish the two. However, if you feel strongly about this then I can revert the change. lmk.

I'll treat this as priority today, would be nice to get this merged soon.

[Vaibhavs10]

Note: that text-to-speech is an alias to text-to-audio pipeline so the behaviour would be exactly the same for both.

[osanseviero]

Ok from my side to follow-up with the icon, but as mentioned before, we should reuse the widget, no need to implement a new one

[Vaibhavs10]

@osanseviero - Removed all the custom TextToAudio code. I am waiting for the checks to run.

Is there anything else left for this to be good to merge? @mishig25 @osanseviero

[Vaibhavs10]

Both the checks have passed! 🤗

[osanseviero]

LGTM to merge if the TTA Inference API is working already

[Vaibhavs10]

Good to merge, then?

For testing the inference API, feel free to head over to this URL: https://huggingface.co/reach-vb/musicgen-small

[mishig25]

if it is exact same icon as IconTextToSpeech, why not just use IconTextToSpeech ?

[Vaibhavs10]

Good call: I fixed it in the latest commit.

[osanseviero]

It seems stuck in loading or internal server errors though

[Vaibhavs10]

@osanseviero -> The model is pretty much unusable on CPU (it takes an insane amount of time to generate).

I tested it from api-inference main - It returns a file as expected!

The plan is to enable these for GPU run-time once the pipeline is merged.