Generate text until condition

[StellaAthena]

Is there a simple way to have the model generate text until a condition is met? I'm interested in data memorization and want to prompt the model with some tokens from the training data and then have it generate text until it makes a mistake (aka deviates from the training data). The naive approach with a while loop has significant overhead, and I was wondering if there was something smarter I can be doing.

[LysandreJik]

Maybe of interest to @patrickvonplaten @patil-suraj

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[patrickvonplaten]

Related PR: #12219