Make a tokens_tortl() and tokens_toltr()

[koheiw]

We already have tokens_tortl():
ropensci/textworkshop18#16

tokens_toltr() is only to remove the direction marker "\u200F". We can also add rtl or direction argument to tokens() or corpus() to make this functionality more visible.

[koheiw]

I am actually not sure if tokens_toltr() should only remove the direction marker or replace it with "\u200E".

Direction makers pass through tokenization, so it might be easier to do in corpus(rtl = TRUE). If we want to keep texts intact in corpus objects, tokens(rtl = TRUE) is the answer.

[koheiw]

To avoid modifying tokens, we should insert the direction mark after punctuations in print.tokens() and print.dfm(), but this means that we will add direction attribute/slot to these objects.

> dfm(toks[1])
Document-feature matrix of: 1 document, 6 features (0% sparse).
1 x 6 sparse Matrix of class "dfm"
       features
docs    מדינת ישראל נוסדה בשנת 1948 .
  text1     1     1     1    1    1 1
> dfm(tokens_tortl(toks[1]))
Document-feature matrix of: 1 document, 6 features (0% sparse).
1 x 6 sparse Matrix of class "dfm"
       features
docs    מדינת ישראל נוסדה בשנת 1948 .‏
  text1     1     1     1    1    1 1