"Storage - An ocean of bits & bytes"

This codebase is my submission for the hackathon conducted by Cloudwiry, titled 'Storage - An ocean of bits & bytes'.

Video Link:

Click here for the video demo, and explanation.

Architecture:

Stack:

1. AWS EFS - Distributed storage disk
2. AWS RDS (postgres) - User data storage
3. AWS Elastic Beanstalk - Managed servers, Application Load Balancing
4. AWS Route53 - DNS, Routing
5. AWS Codepipeline, Cloudwatch - CICD
6. FastAPI - Web framework

Description:

In this solution, users can interact with a FastAPI application, which acts as a blob storage system. The core logic of my solution is that an AWS EFS volume, which is automatically mounted to EC2 instances as a part of AWS Elastic Beanstalk environment, will serve as a distributed storage disk. The Beanstalk environment also has an Application Load Balancer. Users from multiple availbility zones can access the same storage system (The EFS volume), even if they are routed to different servers(EC2 Instances) by the Application Load Balancer. An AWS RDS postgres instance stores the usernames, encrypted passwords, names and IDs of the users.

Key Features:

• User Account creation/deletion
• User Authentication, Password Encryption and API Protection
• Upload, Rename, Share, Download, Delete file functionalities

Bonus features:

• Trash, which stores deleted files until emptied, for every user
• Autoscaling, Load Balancing and Distributed file storage
• CICD for faster bug fixes and quicker releases
• File compression

Installation

This application has been deployed to AWS Beanstalk.

Instructions:

All you need to do is download the postman collection file and start testing out the APIs. However, if you feel the need to run the server locally:

1. Clone this repository

git clone https://github.com/Vishakhavel/file-management-system.git

2. Move into the project directory

cd file-management-system

3. Create a virtual environment

python3 -m venv <your-venv-name>

4. Activate the virtual environment

source <your-venv-name>/bin/activate

5. Install dependencies

pip3 install -r requirements.txt

6. Start the server

uvicorn application:app --reload

Usage

Download this Postman collection file, open it in postman and use it to test the application, after creating the environment and the environment variables as explained in the video.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Please make sure to update tests as appropriate.