time-series-analysis

Web app which takes excel or csv parse and clean data ,store it in database and analyze it, and display interactive charts with time delta selection functionality.

Run app using docker Setup

Clone the repository (if not already done):

git clone <repo-url>
cd time-series-analysis

2. Build the Docker images: In the root of the project, run the following command to build both the backend and frontend images.

docker-compose build

3. Run the containers: Once the build is successful, start the containers using Docker Compose:

docker-compose up

This will start the backend on port 8000 and the frontend on port 3000.

4. Access the application: Frontend: Open a browser and go to http://localhost:3000 to view the React frontend. Backend: The FastAPI backend will be running on http://localhost:8000.
5. Clean up (optional): To stop and remove the containers:

docker-compose down

High level Architechture

1. Every file uploaded is a single table in the db

2. Currently only store the cleaned data to postgres db

3. We fetch the whole dataframe into a grid to preview the cleaned data stored

4. We fetch weekly,monthly,yearly aggregations (i.e mean,median and count for each delta) and plot it into line chart in the similar chart like shown below

5. Apart from the aggregations, there is an option for the user to pick dates range and the selected time range plot is shown as well

Run the app locally using the code

Clone the repository (if not already done):

git clone <repo-url>
cd time-series-analysis

For Backend

Make sure your postgre server is already running

cd backend

Activate your favorite virtual env, in my case it was conda env

conda activate time-series-analysis

Install all requirements in the environment

pip install -r requirements.txt

Run main.py file in any of your favorite editor

For Frontend

Make sure backend server is running successfully, then only the api types are generated in FE.

cd frontend
pnpm install
pnpm start:dev

Setup db based on machine: Attaching here for macos

Install postgresql using brew,

use @17 for compatibility of psycopg2 and timescaledb versions in the requirements file

brew install postgresql@17

Install timescaledb-tune 0.18.0 (darwin amd64)

use this documentation https://docs.timescale.com/self-hosted/latest/install/installation-macos/#install-and-configure-timescaledb-on-postgresql

Add the TimescaleDB extension to your database

make sure u login to the psql cli using

psql -U $(whoami) -d postgres

Then follow steps here https://docs.timescale.com/self-hosted/latest/install/installation-macos/#add-the-timescaledb-extension-to-your-database

Some commonly used commands throughout

• To list users \du, list dbs \l, list extensions \dx , quit \q
• To grant access to a user GRANT ALL PRIVILEGES ON DATABASE <dbname> TO <user>;

Linking libq for psycopg2

export PATH="/opt/homebrew/opt/libpq/bin:$PATH"
export LDFLAGS="-L/opt/homebrew/opt/libpq/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libpq/include"
export PKG_CONFIG_PATH="/opt/homebrew/opt/libpq/lib/pkgconfig"
export DYLD_LIBRARY_PATH="/opt/homebrew/opt/libpq/lib:$DYLD_LIBRARY_PATH"