The Distributed Task Queue is a web application that allows users to create tasks and process them asynchronously with prioritization. The application uses Flask, MongoDB, and Redis to efficiently distribute the tasks to workers, which execute them based on their priority.
- Task creation with priority (low, medium, or high)
- Asynchronous task processing using workers
- Task monitoring and status updates
- Priority-based task processing
- Pagination, sorting, and filtering tasks
- Flask: A lightweight web framework for Python, used to create the web application.
- MongoDB: A NoSQL database used to store tasks.
- Redis: An in-memory data store used as a message broker for the task queue.
- RQ (Redis Queue): A simple Python library for queueing jobs and processing them in the background with workers, built on top of Redis.
- Flask-PyMongo: A Flask extension that provides integration with MongoDB.
- Clone the repository:
git clone https://github.com/eskabore/distributed_task_queue.git- Change to the project directory:
cd distributed_task_queue- Create a virtual environment:
python -m venv venv- Activate the virtual environment:
- On Windows:
venv\Scripts\activate - On macOS and Linux:
source venv/bin/activate
- Install the required packages:
pip install -r requirements.txt- Set the Flask environment variables:
- On Windows:
set FLASK_APP=app set FLASK_ENV=development
- On macOS and Linux:
export FLASK_APP=app export FLASK_ENV=development
- Start the development server:
flask run- Open your browser and navigate to
http://localhost:5000/.
- Create a new task by clicking on "Create Task" and filling out the form.
- Monitor the status of your tasks by clicking on "View Tasks".
- View the details of a specific task by clicking on the task ID.
This project is licensed under the MIT License.