Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read.py Prompt Engineering Error #65

Closed
nulflux opened this issue May 25, 2022 · 6 comments
Closed

Read.py Prompt Engineering Error #65

nulflux opened this issue May 25, 2022 · 6 comments

Comments

@nulflux
Copy link

nulflux commented May 25, 2022

Attempting to use prompt engineering with Read.py leads to this error:

Traceback (most recent call last):
File "tortoise/beyond.py", line 68, in
gen = tts.tts_with_preset(text, voice_samples=voice_samples, conditioning_latents=conditioning_latents,
File "/media/5TB/Env/TortoiseTTS/tortoise-tts/tortoise/api.py", line 302, in tts_with_preset
return self.tts(text, **kwargs)
File "/media/5TB/Env/TortoiseTTS/tortoise-tts/tortoise/api.py", line 451, in tts
wav_candidates = [potentially_redact(wav_candidate, text) for wav_candidate in wav_candidates]
File "/media/5TB/Env/TortoiseTTS/tortoise-tts/tortoise/api.py", line 451, in
wav_candidates = [potentially_redact(wav_candidate, text) for wav_candidate in wav_candidates]
File "/media/5TB/Env/TortoiseTTS/tortoise-tts/tortoise/api.py", line 449, in potentially_redact
return self.aligner.redact(clip.squeeze(1), text).unsqueeze(1)
File "/home/beyond/.local/share/virtualenvs/TortoiseTTS-3YyTW6Mb/lib/python3.8/site-packages/TorToiSe-2.4.0-py3.8.egg/tortoise/utils/wav2vec_alignment.py", line 141, in redact
File "/home/beyond/.local/share/virtualenvs/TortoiseTTS-3YyTW6Mb/lib/python3.8/site-packages/TorToiSe-2.4.0-py3.8.egg/tortoise/utils/wav2vec_alignment.py", line 105, in align
AssertionError: Something went wrong with the alignment algorithm. I've dumped a file, 'alignment_debug.pth' toyour current working directory. Please report this along with the file so it can get fixed.

Is there any fix for this?
alignment_debug.pth.zip

@nulflux
Copy link
Author

nulflux commented May 25, 2022

Apparently I have two different folders with TTTS - here's the most current debug file.
alignment_debug.pth.zip
)

@neonbjb
Copy link
Owner

neonbjb commented May 25, 2022

I'll need to do some debugging, I may get some time tomorrow night to take a look.

@jd-3d
Copy link

jd-3d commented May 25, 2022

I also get this error when prompt engineering. I think it broke with the 5/12 update (which is when I first noticed it).

@nulflux
Copy link
Author

nulflux commented May 26, 2022

Last night I did some testing: prompt engineering does work in some scenarios and not in others. For instance a text file with a single line and an engineered prompt works every time I tested it. I can add up to 4 on the same line. If I go beyond 4 it has an alignment error. I also tested 3 on one line, 3 on the second line - it errors after the 4th. I tested with | dividing the phrases and same issue. However - if I grab a solid paragraph from a normal book and shove it into the text file it works like a champ. It seems to handle other types of braces just fine (){}** etc. yet that doesn't achieve the desired result - it will 'say' the modifier instead of using it - although it does actually add the proper emphasis.

@neonbjb
Copy link
Owner

neonbjb commented May 27, 2022

@space-pope's fix does appear to work from the debugging info I have. Thanks!

@neonbjb neonbjb closed this as completed May 27, 2022
@nulflux
Copy link
Author

nulflux commented May 27, 2022

Cool. Thanks guys :) I'll see if there's anything I can contribute to this as I test things out. For right now it's pretty sweet. Nice work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants