Twitter News

Application that converts the tweets from the trending topics into news

Warning 08/01/2019: This application is not working since the update of the Twitter UI. It needs to be adapted to support it.

---

Client-Server

This application is divided in 2 parts:

• Client: the responsable of showing up the created news in a pretty way.
• Server: the responsable of gathering the necessary information to generate the news and then create them.

How does it works? (the technical part)

The server uses Selenium to go through the trending topics of Twitter. Then, using web scrapping, it obtains all the information of each tweet of each trending topic. This information is processed, the hashtags, links and other relevant things are extracted to have a pure text that can be used to generate the news. After the text analysis is done, the creation of the news starts. The text of the tweets of each trending topic are joined to create a big text that will be used as the base from where the program will create the news. Using the Markovify's implementation of Markov Chains and spaCy a Natural Language Processing software, sentences are created and putted together to form the news. Finally, a JSON file is created with the news of each trending topic.

The client gets the JSON and using React with Semantic UI it shows the news in a web page in a pretty way.

Configuring the application before start using it

After download the application you have to install the necessaries things to run it.

For the Client:

1. Install Node JS
2. Inside the client folder run: npm install

For the Server:

1. Download the chrome driver from here and set an environment variable for it
2. Install Python 3
3. Install pip (probably it's already installed because of the instalation of Python)
4. Install virtualenv: pip install virtualenv
5. Go into the server folder of the project and run:

• virtualenv venv
• source venv/bin/activate

6. Install the necessary dependencies running: pip install -r requirements.txt

How to use the app

Requirement: A Twitter account

Server: To run the server go into the src folder inside the server folder and run

python3 start.py username password

Note: The username and the password are the username (or mail) and password of the Twitter account.

Client: To run the client go to the client folder and run

npm start

Note: Should not be necessary to manually run it. The client is automatically called after the server finishes its work.

---

License

MIT