Add support for other languages than English in get_sentiment for NRC…

[denrou]

The NRC lexicon supports many languages and should partialy solve the issue #13 .
sysdata.rda has been moved to data-raw/sysdata-old.rda and a new sysdata.rda has been created with the new NRC lexicon. Details about how to get this new lexicon is stored in data-raw/data.R

Some changes have been introduced to get sentiment analysis work with multiple languages. In particular, I used dplyr and tidyr depencies to make the dataframe more tidy.
Because the NRC lexicon is quite large, every word with a sentiment set to 0 has been removed.
The data frame is now around 500,000 observations (10 times smaller than the original one).

[mjockers]

This looks like a great addition, thanks. Before I can merge it, will need to think about how to update get_sentences() function to include a "language" argument. get_sentences() implements openNLP which has different language models. From the openNLP R package docs: "For languages other than English, these can conveniently be made available to R by installing the respective openNLPmodels.language package from the repository at http://datacube.wu.ac.at" Looks like openNLP has models for da, de, en, es, it, nl, pt, and sv, which is != to all of the languages in the NRC lexicon, so need to deal with that as well.

[denrou]

Thanks for willing to take the PR into account. I agree that if other languages are integrated in sentiment analysis, they should also be integrated with the other functions of the package.
How would you like to handle the fact that openLNPmodels.language and NRC lexicon can't work with common languages?

[strategist922]

Hi Denrou,
I check the code "https://github.com/denrou/syuzhet/blob/master/data-raw/data.R", there are total 41 language in "http://www.saifmohammad.com/WebDocs/NRC-Emotion-Lexicon-v0.92-InManyLanguages-web.xlsx" May I know the reason why you only inport some of the languages list in NRC-Emotion-Lexicon-v0.92-InManyLanguages-web.xlsx instead of import all the languages in NRC-Emotion-Lexicon-v0.92-InManyLanguages-web.xlsx ?
Ref.
code "https://github.com/denrou/syuzhet/blob/master/data-raw/data.R"
lang_ascii <- c("basque", "catalan", "danish", "dutch", "english", "esperanto", "finnish", "french", "german", "irish", "italian", "latin", "portuguese", "romanian", "somali", "spanish", "sudanese", "swahili", "swedish", "turkish", "welsh", "zulu")

[denrou]

Hi strategist922,

There are indeed other languages that could be imported. The problem I faced was that this other languages had a different encoding and get_nrc_value function crashed when those languages were included. I didn't spend much time on it, so maybe there is a quick solution to fix it.

[hichemmkhalyd]

Hi denrou!

I tried to use get_nrc_sentiment with lang = "portuguese" and it aways returns zero for all sentiments. I was wondering if I shoud do something more to use it, like inport a lexicon in portuguese or something else.

[mjockers]

merged and revised arc related functions for efficiency in v 1.0.3 and for expandability with user defined lexicons.