Infinite recursion on TokenizedStrings with mismatched brackets on function calls

[markfinal]

If there are mismatched brackets on function calls, e.g.

@dir($(0)

then this resulted in an infinite loop, because the balancing groups on the regex always returned no matches, feeding the expression containing the function back into the parsing code.