Frog Freeze

[tisvonkje]

frog (0.13.6) freezes on input that contains large sequences of exclamation marks. To reproduce, create a file input.txt with the following content:
Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
run:
frog -t input.text

[proycon]

After some investigation, this issue is caused by ucto rather than Frog itself:

[tokenizeLine] punctuation or digit detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] punctuation or digit detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] punctuation or digit detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] punctuation or digit detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] punctuation or digit detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] punctuation or digit detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] space detected, word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
[tokenizeLine] Further tokenisation necessary for: [Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
   [tokenizeWord] Input: (45) word=[Nee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!]
        TESTING GENITIVEPLACENAME
        TESTING WORD-WITHSUFFIX
        TESTING WORD-INFIX-COMPOUND
        TESTING QUOTE-SUFFIX
        TESTING WORD-DIMIN
        TESTING WORD-TOKEN
        TESTING NUMBER-ORDINAL
        TESTING DATE-REVERSE
        TESTING DATE
        TESTING URL
        TESTING URL-WWW
        TESTING URL-DOMAIN
        TESTING E-MAIL
        TESTING ABBREVIATION-COMPOUND
        TESTING COMPOUND-ABBREVIATION
        TESTING QUOTE-COMPOUND
        TESTING WORD-COMPOUND
        TESTING NUMBER-COMPOUND
        TESTING INITIAL
        TESTING ABBREVIATION
        TESTING SMILEY
        TESTING REVERSE-SMILEY
        TESTING ABBREVIATION-KNOWN
        TESTING CURRENCY
        TESTING NUMBER-STRING
        TESTING STRING-NUMBER
        TESTING WORD-PARPREFIX-PARSUFFIX
        TESTING WORD-PARPREFIX
        TESTING WORD-PARSUFFIX
        TESTING INITIALS
        TESTING PUNCTUATION-MULTI-DOT

Hangs indefinitely after this...

[kosloot]

solved in Ucto