prismaudio 401 Client Error

[EvieMu]

When I followed the Quick Start and Run Demo procedure, an error occurred at the final step：
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/google/t5gemma-l-l-ul2-it/resolve/main/config.json

[lkcqswb]

Hi, this is because google/t5gemma-l-l-ul2-it is a gated model. You need to go to https://huggingface.co/google/t5gemma-l-l-ul2-it and fill in the form to request access. Once approved, set your HF token as an environment variable:

export HF_TOKEN="your_token_here"

That should fix it!

[EvieMu]

Thank you, This issue has been resolved.But, I got a new problem
FileNotFoundError: [Errno 2] No such file or directory: 'ThinkSound/configs/model_configs/prismaudio.json'

[lkcqswb]

Thank you, This issue has been resolved.But, I got a new problem FileNotFoundError: [Errno 2] No such file or directory: 'ThinkSound/configs/model_configs/prismaudio.json'

Sorry about that, this is a bug and we'll push a fix shortly. As a workaround, you can replace ThinkSound/configs/model_configs/prismaudio.json with PrismAudio/configs/model_configs/prismaudio.json for now.

[EvieMu]

Thank you. I have successfully run a demo.
I experienced prismaudio through HuggingFace a few hours ago, and the result is very good. But now, I use my same video and prompt through local deployment, the result is so bad. Is there any difference between the two forms?

[lkcqswb]

Sorry for the inconsistency! The local model is identical to the HuggingFace demo — the quality difference is likely due to randomness in the generation process. Try running it a few more times, as results can vary.

[EvieMu]

How can I set the seed? The current seed is fixed at 42

[lkcqswb]

How can I set the seed? The current seed is fixed at 42

Thank you for your patience. After carefully reviewing the local code, the random seed is indeed fixed. However, the quality difference may stem from the HuggingFace Space being a continuously running service — each user request introduces different randomness into the generation process. We apologize for any inconvenience this may have caused. If you'd like better generation results, try using a more detailed and descriptive prompt, which can often lead to significant improvements.

[EvieMu]

my demo command is : ./scripts/PrismAudio/demo.sh input_videos/woman-seaside.mp4 "The waves are lapping against the beach."
I guess the description didn't work. Is there any problem with my description?

[lkcqswb]

my demo command is : ./scripts/PrismAudio/demo.sh input_videos/woman-seaside.mp4 "The waves are lapping against the beach." I guess the description didn't work. Is there any problem with my description?

You may want to kindly check if your input is automatically included in cot_coarse/cot.csv. Additionally, we would warmly encourage more detailed descriptions — perhaps you could consider using an LLM to help elaborate your input further, and see if the generated results improve.

[EvieMu]

When I run a demo, my description indeed appears in cot_coarse/cot.csv. Although I used the same video, same seed and different description, the result audio sounds like there hasn't been any change. So, I guess the description didn't work. Please check it.

[lkcqswb]

When I run a demo, my description indeed appears in cot_coarse/cot.csv. Although I used the same video, same seed and different description, the result audio sounds like there hasn't been any change. So, I guess the description didn't work. Please check it.

Sorry, I think I know the cause of the bug. We will fix it in the code. The bug is caused by the fact that running demo.sh will compress your input into results/demo.npz, however due to our oversight, when you change the input it will be skipped because demo.npz already exists. I think a quick fix is to add a line rm results/demo.npz at the end of demo.sh, or delete results/demo.npz after each run. We sincerely apologize for any inconvenience caused.