Skip to content

Flask app that displays articles scraped from web. Using Beautiful Soup 4, and request for web scrapping, Celery scheduling tasks, and PostgreSQL as the database.

License

Notifications You must be signed in to change notification settings

atahanyorganci/dailyspokesman

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

75 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DailySpokesMan

DailySpokesMan (DSM) is server side rendered web app that serves a local copy of Turkish newspaper Sözcü.

Table of Contents

About

DailySpokesMan (DSM) is an web app that serves a modified local copy of Sözcü newspaper. DSM is a personal project for teaching myself full-stack development by building a Flask server that serves both a REST API, and frontend views.

Features

  • DSM automatically stores news articles from Sözcü newspaper in SQL-Lite database.
  • REST API for querying articles, and state of the database
  • Serverside rendered pages

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

How to Install

Make sure you have installed Python 3.7.0 or higher with Pipenv package and Node.js with npm package manager. After cloning the repository cd into it run following commands.

  • Create virtual environment using Pipenv.
$ pipenv shell # create virtual enviroment
  • Install dependencies to the virtual environment.
$ pipenv install # install dependencies
  • After installing the dependencies initialize the SQL-Lite database
$ export FLASK_APP=run.py
$ flask db init
$ flask db migrate -m "initialize database"
$ flask db upgrade

Now all the dependencies are installed, and database has been initialized complete.

  • Install Redis server
$ .scripts/redis.sh

Usage

  • Run Celery worker
$ .scripts/celery.sh
  • Run task scheduler provided by Celery
$ .scripts/scheduler.sh
  • Run the web server
$ .scripts/run.sh

Built Using

  • Pipenv for handling Python dependencies.
  • Flask, micro web development framework for Python.
  • SQL-Alchemy Python wrapper around SQL databases, this project uses SQL-Lite.
  • Beautiful Soup 4, used for web scrapping, gathering news from Sözcü newspaper.
  • Flask Migrate for migrations.
  • Celery for scheduling, and handling web scrapping tasks.

Authors

About

Flask app that displays articles scraped from web. Using Beautiful Soup 4, and request for web scrapping, Celery scheduling tasks, and PostgreSQL as the database.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published