special symbols in text

[152334H]

Notable problem of the base tortoise model: it does not handle *emphasis* or special symbols like + or = or & elegantly. Lines like:

• 8 + 5 = 13
• him & his wife
• I *really* do not like this

all perform oddly poorly.

I am not sure if fine-tuning could fix this. Personally, I think you would need to at least fine-tune both the CLVP && GPT model to fix this.

[xenotropic]

This seems like it would be easier to fix with text preprocessing / normalization (for plus, equals, ampersand) and bold emphasis with bracket prompt engineering, "I [emphatically] do not like this".

[152334H]

yeah my existing hobby projects all do that kind of thing, but there's no inherent reason why transformers shouldn't be able to just handle the symbols