-
Notifications
You must be signed in to change notification settings - Fork 39
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
read from txt and actually expand lexicon from synonyms
- Loading branch information
1 parent
a8be2a2
commit 355a6cb
Showing
4 changed files
with
402 additions
and
212 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,88 +1,89 @@ | ||
| # NRCLex | ||
|
|
||
| (C) 2019 Mark M. Bailey, PhD | ||
|
|
||
| ## About | ||
| NRCLex will measure emotional affect from a body of text. Affect dictionary contains approximately 27,000 words, and is based on the National Research Council Canada (NRC) affect lexicon (see link below) and the NLTK library's WordNet synonym sets. | ||
|
|
||
| Lexicon source is (C) 2016 National Research Council Canada (NRC) and this package is **for research purposes only**. Source: http://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm As per the terms of use of the NRC Emotion Lexicon, if you use the lexicon or any derivative from it, cite this paper: Crowdsourcing a Word-Emotion Association Lexicon, Saif Mohammad and Peter Turney, Computational Intelligence, 29 (3), 436-465, 2013. | ||
|
|
||
| NLTK data is (C) 2019, NLTK Project. Source: [NLTK] (https://www.nltk.org/). Reference: Bird, Steven, Edward Loper and Ewan Klein (2009), Natural Language Processing with Python. O’Reilly Media Inc. | ||
|
|
||
| ## Update | ||
| * Finally got around to cleaning this up a bit. Updated PyPI package with current version. Thanks to all the contributors for cleaning up my terrible code! | ||
| * Expanded NRC lexicon from approximately 10,000 words to 27,000 based on WordNet synonyms. | ||
| * Minor bug fixes. | ||
| * Contributor updated NTC library. | ||
|
|
||
| ## Installation | ||
| `pip install NRCLex` | ||
|
|
||
| ## Affects | ||
| Emotional affects measured include the following: | ||
|
|
||
| * fear | ||
| * anger | ||
| * anticipation | ||
| * trust | ||
| * surprise | ||
| * positive | ||
| * negative | ||
| * sadness | ||
| * disgust | ||
| * joy | ||
|
|
||
| ## Sample Usage | ||
|
|
||
| `from nrclex import NRCLex`<br><br> | ||
|
|
||
|
|
||
| *#Instantiate NRCLex object, you can pass your own dictionary filename in json format.*<br> | ||
|
|
||
| `text_object = NRCLex(lexicon_file='nrc_en.json')`<br><br> | ||
|
|
||
|
|
||
| *#You can pass your raw text to this method(for best results, 'text' should be unicode).*<br> | ||
|
|
||
| `text_object.load_raw_text(text: str)`<br><br> | ||
|
|
||
|
|
||
| *#You can pass your already tokenized text as a list of tokens, if you want to use an already tokenized input. | ||
| This usage assumes that the text is correctly tokenized and does not make use of TextBlob.*<br> | ||
|
|
||
| `text_object.load_token_list(list_of_tokens: list)`<br><br> | ||
|
|
||
|
|
||
| *#Return words list.*<br> | ||
|
|
||
| `text_object.words`<br><br> | ||
|
|
||
|
|
||
| *#Return sentences list.*<br> | ||
|
|
||
| `text_object.sentences`<br><br> | ||
|
|
||
|
|
||
| *#Return affect list.*<br> | ||
|
|
||
| `text_object.affect_list`<br><br> | ||
|
|
||
|
|
||
| *#Return affect dictionary.*<br> | ||
|
|
||
| `text_object.affect_dict`<br><br> | ||
|
|
||
|
|
||
| *#Return raw emotional counts.*<br> | ||
|
|
||
| `text_object.raw_emotion_scores`<br><br> | ||
|
|
||
|
|
||
| *#Return highest emotions.*<br> | ||
|
|
||
| `text_object.top_emotions`<br><br> | ||
|
|
||
|
|
||
| *#Return affect frequencies.*<br> | ||
|
|
||
| `text_object.affect_frequencies` | ||
| # NRCLex | ||
|
|
||
| (C) 2019 Mark M. Bailey, PhD | ||
|
|
||
| ## About | ||
| NRCLex will measure emotional affect from a body of text. Affect dictionary contains approximately 27,000 words, and is based on the National Research Council Canada (NRC) affect lexicon (see link below) and the NLTK library's WordNet synonym sets. | ||
|
|
||
| Lexicon source is (C) 2016 National Research Council Canada (NRC) and this package is **for research purposes only**. Source: http://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm As per the terms of use of the NRC Emotion Lexicon, if you use the lexicon or any derivative from it, cite this paper: Crowdsourcing a Word-Emotion Association Lexicon, Saif Mohammad and Peter Turney, Computational Intelligence, 29 (3), 436-465, 2013. | ||
|
|
||
| NLTK data is (C) 2019, NLTK Project. Source: [NLTK] (https://www.nltk.org/). Reference: Bird, Steven, Edward Loper and Ewan Klein (2009), Natural Language Processing with Python. O’Reilly Media Inc. | ||
|
|
||
| ## Update | ||
| * Finally got around to cleaning this up a bit. Updated PyPI package with current version. Thanks to all the contributors for cleaning up my terrible code! | ||
| * Expanded NRC lexicon from approximately 10,000 words to 27,000 based on WordNet synonyms. | ||
| * Minor bug fixes. | ||
| * Contributor updated NTC library. | ||
|
|
||
| ## Installation | ||
| `pip install NRCLex` | ||
|
|
||
| ## Affects | ||
| Emotional affects measured include the following: | ||
|
|
||
| * fear | ||
| * anger | ||
| * anticipation | ||
| * trust | ||
| * surprise | ||
| * positive | ||
| * negative | ||
| * sadness | ||
| * disgust | ||
| * joy | ||
|
|
||
| ## Sample Usage | ||
| [](https://colab.research.google.com/github/metalcorebear/NRCLex/example.ipynb) | ||
|
|
||
| `from nrclex import NRCLex`<br><br> | ||
|
|
||
|
|
||
| *#Instantiate NRCLex object, you can pass your own dictionary filename in json format.*<br> | ||
|
|
||
| `text_object = NRCLex(txt_file='Spanish-NRC-EmoLex.txt')`<br><br> | ||
|
|
||
|
|
||
| *#You can pass your raw text to this method(for best results, 'text' should be unicode).*<br> | ||
|
|
||
| `text_object.load_raw_text(text: str)`<br><br> | ||
|
|
||
|
|
||
| *#You can pass your already tokenized text as a list of tokens, if you want to use an already tokenized input. | ||
| This usage assumes that the text is correctly tokenized and does not make use of TextBlob.*<br> | ||
|
|
||
| `text_object.load_token_list(list_of_tokens: list)`<br><br> | ||
|
|
||
|
|
||
| *#Return words list.*<br> | ||
|
|
||
| `text_object.words`<br><br> | ||
|
|
||
|
|
||
| *#Return sentences list.*<br> | ||
|
|
||
| `text_object.sentences`<br><br> | ||
|
|
||
|
|
||
| *#Return affect list.*<br> | ||
|
|
||
| `text_object.affect_list`<br><br> | ||
|
|
||
|
|
||
| *#Return affect dictionary.*<br> | ||
|
|
||
| `text_object.affect_dict`<br><br> | ||
|
|
||
|
|
||
| *#Return raw emotional counts.*<br> | ||
|
|
||
| `text_object.raw_emotion_scores`<br><br> | ||
|
|
||
|
|
||
| *#Return highest emotions.*<br> | ||
|
|
||
| `text_object.top_emotions`<br><br> | ||
|
|
||
|
|
||
| *#Return affect frequencies.*<br> | ||
|
|
||
| `text_object.affect_frequencies` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| { | ||
| "nbformat": 4, | ||
| "nbformat_minor": 0, | ||
| "metadata": { | ||
| "colab": { | ||
| "provenance": [] | ||
| }, | ||
| "kernelspec": { | ||
| "name": "python3", | ||
| "display_name": "Python 3" | ||
| }, | ||
| "language_info": { | ||
| "name": "python" | ||
| } | ||
| }, | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "#Prepare dependencies and files\n", | ||
| "\n", | ||
| "Lexicon source is (C) 2016 National Research Council Canada (NRC) and this package is **for research purposes only**. Source: http://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm As per the terms of use of the NRC Emotion Lexicon, if you use the lexicon or any derivative from it, cite this paper: Crowdsourcing a Word-Emotion Association Lexicon, Saif Mohammad and Peter Turney, Computational Intelligence, 29 (3), 436-465, 2013." | ||
| ], | ||
| "metadata": { | ||
| "id": "0cMm__BlMaid" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": { | ||
| "id": "aGocXvXH68jN" | ||
| }, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!pip -q install 'https://codeload.github.com/rivanfebrian123/NRCLex/zip/refs/heads/master'\n", | ||
| "!curl -so 'nrc.zip' 'http://saifmohammad.com/WebDocs/Lexicons/NRC-Emotion-Lexicon.zip'\n", | ||
| "!unzip -qn 'nrc.zip'\n", | ||
| "\n", | ||
| "import nltk\n", | ||
| "nltk.download('punkt')\n", | ||
| "nltk.download('omw-1.4')\n", | ||
| "nltk.download('wordnet')" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "#Let's go!\n", | ||
| "\n", | ||
| "You can also try some other txt files, for example: `Spanish-NRC-EmoLex.txt`" | ||
| ], | ||
| "metadata": { | ||
| "id": "knsMsT0OMvJ9" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "from nrclex import NRCLex\n", | ||
| "\n", | ||
| "pos = 0\n", | ||
| "neg = 0\n", | ||
| "text = \"she's always arguing!\"\n", | ||
| "\n", | ||
| "nrc = NRCLex('NRC-Emotion-Lexicon/NRC-Emotion-Lexicon-Wordlevel-v0.92.txt')\n", | ||
| "emotions = nrc.load_raw_text(text).affect_frequencies\n", | ||
| "\n", | ||
| "emotions" | ||
| ], | ||
| "metadata": { | ||
| "id": "8j01RB23Mj4I" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "#Messing around with sentiments" | ||
| ], | ||
| "metadata": { | ||
| "id": "quNcWFJwOFWT" | ||
| } | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "pos += emotions['positive']\n", | ||
| "pos += emotions['joy']\n", | ||
| "pos += emotions['trust']\n", | ||
| "pos += emotions['anticipation']\n", | ||
| "\n", | ||
| "neg += emotions['negative']\n", | ||
| "neg += emotions['anger']\n", | ||
| "neg += emotions['disgust']\n", | ||
| "neg += emotions['positive']\n", | ||
| "\n", | ||
| "total = pos + neg\n", | ||
| "ratio = abs(pos - neg) / total if total else 0\n", | ||
| "\n", | ||
| "if ratio < .75:\n", | ||
| " print('neutral')\n", | ||
| "elif pos > neg:\n", | ||
| " print('positive')\n", | ||
| "else:\n", | ||
| " print('negative')" | ||
| ], | ||
| "metadata": { | ||
| "id": "FzXDtgrzOLWp" | ||
| }, | ||
| "execution_count": null, | ||
| "outputs": [] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "#There you go!" | ||
| ], | ||
| "metadata": { | ||
| "id": "0DrxsX0WPmct" | ||
| } | ||
| } | ||
| ] | ||
| } |
Oops, something went wrong.