Tiny Cloud

backend-part -----> frontend-part here

Web application for secure cloud file storage.

Upload, store, display, send, download and rename your files.

Topics:

Application structure

The application consists of two parts: backend and frontend. Each part is placed into a separate repository.

Frontend (liaksej-tiny-cloud-frontend)

The user side of the application is written in TypeScript and implemented on the Nextjs 14 framework with App Router.

Apart from Nextjs, the following technologies are used to create the user part of the application:

React;
tailwindcss;
Auth.js aka NextAuth.js - flexible and secure solution for web application authentication;
clsx - small utility to create conditions in the className string;
zod - TypeScript schema validator with static type inference;
use-debounce - react-hook for debounce search;
sharp - application for converting images for Next.js standalone builds;
cypress - E2E testing application;
pnpm - package manager;
docker

The root directory of the application consists of the following files and directories:

Cypress directory contains tests and settings for E2E testing of the application:

e2e directory contains tests
fixtures directory contains pre-installed fixtures for the tests
integration directory contains scripts for authenticating
support directory contains auxiliary commands and other useful cypress utilities.

Key point:

Cypress does not work correctly with Typescript 5.2 and above, in which the application is written. This problem is described in documentation. To successfully run the tests, you need to:

In the tsconfig.json file, replace "moduleResolution": "bundler" to "moduleResolution": "node" for the duration of the tests.

Have an application (client and server) configured and running;

Have a superuser with email admin@example.com and password 45641231.

Command to run pnpm run cypress:run

Don't forget to return "bundler" in tsconfig.json after running the tests.

Public directory contains static files (images) for the application UI.

The file hero-desktop.png is used based on the license agreement

Src directory is the main directory of the application, where all of its code base is stored:

The src directory is divided into the following subdirectories:

The app directory is the router of the application.
- directory admin - rout to the administrator panel of the user interface, contains:
  - file page.tsx - page of the administrator panel of the user interface;
  - file layout.tsx - layout of the user interface administrator panel page;
  - file error.tsx - error page when an error occurs on the admin panel;
  - directory [name] - rout to user storage from the administrator panel. It consists of:
    - file page.tsx - user page from the administrator panel;
    - file error.tsx - error page when an error occurs on the admin panel;
    - directory [id]/edit - rout to the user's file attribute editing page from the admin panel;
      - file page.tsx - page for editing user file attributes from the administrator panel;
      - file not_found.tsx - Not Found page in case you go to the page of editing user file attributes from the admin panel, which does not exist;
- directory dashboard - rout to the main control panel of the personal file storage;
  - file page.tsx - personal file storage control panel page;
  - file layout.tsx - layout page of the personal file storage control panel;
  - file error.tsx - error page when an error occurs on the admin panel;
    - directory [id]/edit - rout to the user file attribute editing page;
      - file page.tsx - page for editing user file attributes;
      - file not_found.tsx - Not Found page in case you go to the user file attributes editing page that does not exist;
- directory download/[id] - rout to the endpoint to download the user's private file;
  - file route.ts - route handler to retrieve the user's private file that performs the authentication process.
- directory login - rout to the application's login page;
  - file page.tsx - application login page;
- directory signup - rout to the application's registration page;
  - file page.tsx - application registration page;
- file page.tsx - main page of the application;
- file layout.tsx - layout of the main page of the application with basic font settings;
The lib contains service functions of the client part of the application.
- file actions contains functions responsible for sending data via API, as well as changing this data:
  - function authenticate is responsible for authorization through the login form;
  - function sendFileToServer is responsible for uploading files to the server;
  - function deleteFile is responsible for deleting the file from the server;
  - function deleteUser is responsible for deleting the user;
  - function updateFile is responsible for making changes to the file;
  - function updateAdminStatus is responsible for changing the administrator status of the user;
  - function registrate is responsible for registering the user in the application;
- file data contains functions responsible for obtaining API data:
  - function fetchDataFromAPI is responsible for executing fetch requests;
  - function fetchTableData is responsible for retrieving the list of files / users;
  - function fetchFilesPages is responsible for getting the number of pages for the paginator;
  - function fetchFile is responsible for retrieving data about a particular file;
  - function adminCheck is responsible for additional checking of user access rights (if the user is an administrator, the administrative panel is displayed in the interface);
- file definitions contains descriptions of the types and interfaces of TypeScript objects;
- file utils contains utility functions for pagination, handling the displayed date and file size;
Directory ui contains a set of React components for the application UI;
- directory adminPanel contains components of the administrative panel:
  - component AdminTable contains administrative panel table;
  - component ChangeAdminStatusButton contains checkbox to change the administrator status;
- directory dashboard contains components of the user storage control panel:
  - component Breadcrumbs contains breadcrumbs, navigator to subpages;
  - component Buttons contains the button to delete a file / user;
  - component CopyLinkButton contains the button to copy/pass the public link of the file;
  - component DashboardTable - table of the user's storage control panel;
  - component EditFileForm - file data editing form;
  - component ModalUpload - modal window layout of the file upload window;
  - components from file Pagination - pagination elements on the user store control panel and administration panel;
  - component Search - search field by file name on the user storage control panel / username on the administration panel;
  - component UpdateInfoButton - button to save changes in the file data editing form;
- ветка sideNav contains sidebar components;
  - component AdminLink - button to access the administrative panel;
  - component CloudLogo - application logo;
  - component SideNav - main sidebar UI;
  - components from file UploadFile - file upload button in the sidebar, as well as the contents of the modal window and the file upload modal window button;
- component Button - buttons UI;
- component LoginForm - application login form;
- component RegistrationForm - registration form of the application;
- file fonts defines fonts used in the application's UI;
- file global.css - defines some global application styles that are not defined for Tailwind CSS;
- components from file skeletons - skeletons for user storage control panel / administration panel tables;
file auth.config.ts defines the authentication configuration in the application user interface and the interaction with authentication on the server;
file middleware.ts embeds authentication into the application middleware, which ensures that access rights to each application page are controlled right on the server before the page is sent to the user;

The rest of the files are in the rout of the directory:

.example.env.production - example of environment variable settings for building and running the application;
.dockerignore contains the list of files ignored by Docker during build;
.eslintrc.json - ESLint customization file for Next.js;
.gitignore contains a list of files ignored by Git;
Dockerfile contains commands to build the application container;
cypress.config.ts contains configuration settings for Cypress;
next.config.js contains configuration settings for Next.js;
package.json contains a list of installed packages and project configuration settings, scripts for debugging/testing/building the project;
pnpm-lock.yaml contains service information about installed packages for the pnpm package manager;
postcss.config.js contains the configuration settings for CSS processing at build time;
tailwind.config.ts contains the tailwindcss configuration settings;
tsconfig.json contains the configuration settings for TypeScript;
types.d.ts contains the type refinements for Auth.js needed to enable Auth.js to interoperate with SimpleJWT;

Backend structure (liaksej-tiny-cloud-backend)

The server part of the application is written in Python 3 implemented on the framework Django 5.0 in conjunction with the Django REST framework.

In addition to Django 5.0 and Django REST framework the following technologies are used to create the server part of the application:

Simple JWT - JWT authentication plugin for Django REST Framework;
poetry - package manager for python;
pytest - testing framework;
beautifulsoup4 - HTML parsing library;
docker;
docker-compose;

The root directory of the application consists of the following files and directories:

Application authentication responsible for authentication and administration:

package migrations contains migration modules;
module admin.py contains settings for displaying authentication application elements on the Django administration panel;
module apps.py contains information about the authentication application to be initialized in the settings.py;
module models.py does not contain any additional information;
module serializers.py contains:
- class UserListSerializer, provides data serialization for the /api/auth/users/ endpoint;
- class CustomRegisterSerializer provides serialization of data for user registration, endpoint /api/auth/register/;
module urls.py describes the endpoints of the authentication application;
module views.py describes the following DRF representations of the authentication application:
- class-view LoginView provides the ability to authenticate in the UI of the admin application to the Django admin panel without having to re-authenticate;
- class-view UsersViewSet provides a view of the user list, individual user data, user deletion for the /api/auth/users endpoint;
- class-view CustomRegisterView provides a user registration view for the /api/auth/register/ endpoint;

Application cloud responsible for working with the user's file storage:

package migrations contains migration modules;
module admin.py contains settings for displaying cloud application elements on the Django administration panel;
module apps.py contains information about the cloud application to initialize in the settings.py;
module models.py contains a description of the cloud application models:
- class User describes the user model;
- class File describes the model of the file;
module permissions.py contains additional authorization settings:
- class IsOwner checks if the owner of the file has rights;
- class IsOwnerOrStaff checks if the file owner or administrator has rights;
module serializers.py contains a serializer:
- class FilesListSerializer provides data serialization for file or file list displays, endpoint /api/cloud/files/;
module services.py contains service functions for saving a file to the server and deleting a file from the server;
module urls.py describes the endpoints of the cloud application;
module views.py describes the following DRF representations of the cloud application:
- class-view FileViewSet provides file list view, individual file data, file deletion and file modification for the /api/cloud/files/ endpoint;
- class-view FileDownloadMixin provides correctness of file transfer process to user (owner) for /api/cloud/download/ endpoint;
- class-view DownloadFileView provides the file transfer process to the owner(s) for the /api/cloud/download/ endpoint;
- class-view PublicFileDownloadView provides the process of transferring a public file to the user for the /api/cloud/public/ endpoint;

Package djangoProject contains the basic server settings of the Django framework:

module asgi.py - asynchronous web specification between the server and the application;
module settings.py contains the basic server settings;
module test_settings.py contains corrective server settings for testing;
module urls.py contains the main endpoints of the server;
module wsgi.py - web specification between the server and the application;

Directory nginx contains Nginx settings:

file Dockerfile contains instructions for building the Nginx container;
file nginx.conf contains Nginx settings;

Directory tests Contains unit and integration tests, as well as fixtures and mocs for the server:

directory tests_authentication contains module test_auth_api.py with authentication application integration tests;
directory tests_cloud contains:
- module test_cloud_api.py with cloud application integration tests;
- module test_services.py with unit tests of service functions of the cloud application;

The rest of the files are in the rout of the directory:

.coveragerc contains settings for the coverage package that provides test coverage verification for the application;
.dockerignore contains a list of files ignored by Docker during build;
.example.env - example of environment variable settings for building and running Django server;
.example.env.db - example of environment variable settings for building and running the database;
.gitignore contains a list of files ignored by Git;
Dockerfile contains commands to build the Django server container;
docker-compose.yml - contains settings for launching application containers;
entrypoint.sh - shell script that checks for database startup before starting the Django server container;
manage.py - Django server executable module;
poetry.lock contains service information about installed Poetry package manager packages;
pyproject.toml contains a list of installed packages and application configuration settings for the Poetry package manager;
pytest.ini - settings file for pytest;

Deployment instructions

The application can be deployed on a server that supports docker.

The installation will require about 1.5 GB of free disk space for the project, in addition to the installation space docker and docker-compose.

Deployment process

Software preparation:

Install Docker: https://docs.docker.com/engine/
Install Docker-compose: https://docs.docker.com/compose/install/
Install Git: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
Clone the main branches of the project repositories into one folder so that they are at the same level. For example, ~/tiny-cloud/liaksej-tiny-cloud-backend and ~/tiny-cloud/liaksej-tiny-cloud-frontend.
```
git clone -b main https://github.com/Liaksej/liaksej-tiny-cloud-backend.git

git clone -b main https://github.com/Liaksej/liaksej-tiny-cloud-frontend.git
```

Preparing environment variables:

In the /liaksej-tiny-cloud-backend directory on the server, open the .example.env file. This is the configuration file for the Django server. Adjust the necessary lines to match the desired application settings:

ADMIN_EMAIL="admin@example.com" # Replace with your address or place a blanking plate
ADMIN_NAME="Admin" # Replace with your name or leave it as it is
ALLOWED_HOSTS="web liaksej-tiny-cloud-backend localhost 127.0.0.1 [::1] 80" # Add the hostname of your server.
CORS_ALLOWED_ORIGINS="http://localhost:3000 http://0.0.0.0:3000 http://web http://nextjs" # Add the hostname of your server.
CSRF_TRUSTED_ORIGINS="" # Add the hostname of your server.
DATABASE="postgres"   # !!!Don't change
DATABASE_NAME="liaksej-tiny-cloud-db"
DATABASE_HOST="db" #!!! Don't change
DATABASE_PORT=5432
DATABASE_USER="postgres" # Change the database username, if necessary.
DATABASE_PASSWORD="verystrongpassword123" #Set the database user password.
DEBUG=0
EMAIL_HOST="smtp.example.com" #Install your machine's smtp server or put in a stub.
EMAIL_HOST_USER="admin@example.com" # Set the administrator's name or place a blank.
EMAIL_HOST_PASSWORD="password" #Set the password for the smpt server.
EMAIL_PORT="588" # Set the port of the smtp server
EMAIL_USE_TLS="True"
MEDIA_URL="/media/"
SECRET_KEY="verystrongsicret123" # Set the secret key for Django: python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"
SIGNING_KEY_JWT="verystrongsicretHS256" #Set the JWT key: python -c "import os; print(os.urandom(32))"
STATIC_URL="/static/"

Change the file name from .example.env to .env

In the same directory, edit the .example.env.db file. It is responsible for the postgres database environment variables.

POSTGRES_USER="postgres" # The username must match the username in the .env file
POSTGRES_PASSWORD="verystrongpassword123" # Password must match the user password in .env
POSTGRES_DB="liaksej-tiny-cloud-db" # The database name must match the database name in .env

Change the file name from .example.env.db to .env.db
In the ./nginx folder located in the liaksej-tiny-cloud-backend project bark, open the nginx.conf file. Replace server_name with your hostname:
```
# ...
server {

 listen 80;
 server_name http://yourdomain.name; # Set the hostname of the server

# ...
```

Navigate to the liaksej-tiny-cloud-frontend folder of the cloned repository, open the file .example.env.production and modify the environment variables by setting the required values.

AUTH_SECRET="supersecretkey123" # Set the jwt signature token for authjs `openssl rand -base64 32`
NEXTAUTH_URL="http://yourhostname.com/" # Set your host name.
NEXTAUTH_BACKEND_URL="http://web:8000/api/"
NEXT_PUBLIC_BACKEND_URL="http://web:8000/api/"
NEXT_PUBLIC_HOSTNAME="http://yourhostname.com/" # Set your host name.

Change the file name from .example.env.production to .env.production.
Open the Dockerfile file in the root of liaksej-tiny-cloud-frontend. On line 26, replace http://localhost/ with your hostname.
```
# ...
# Replace http://localhost/ with your hostname.
ENV NEXT_PUBLIC_HOSTNAME "http://localhost/"
# ...
```

Application Build:

Return to the liaksej-tiny-cloud-backen directory. You are ready to start building the project on the server.
While in the liaksej-tiny-cloud-backen directory, run the application build via the command line:
```
# You may need sudo privileges
docker-compose up -d
```

Comments:

If you have trouble with buildx when building the nextjs container and you understand Docker, you can install buildx following the instructions. If you don't have time to figure it out, replace in the Dockerfile located in the liaksej-tiny-cloud-frontend directory with lines 16 and 33:
# Replace line 16
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prod --frozen-lockfile
# With the line:
RUN pnpm install --prod --frozen-lockfile

# Replace line 33
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile
# With the line:
RUN pnpm install --frozen-lockfile
It'll take longer to build. But the result is the same.

Launching the app:

After building the app, run the following commands to apply migrations, build statics for Django, and create a superuser:
```
# Each of the commands may require sudo privileges

# 1. Apply migrations:
docker-compose exec web python manage.py migrate --noinput

# 2. Collect static:
docker-compose exec web python manage.py collectstatic --noinput 

# 3. Create a superuser:
docker-compose exec web python manage.py createsuperuser
```
Important comments

The superuser we created can log into the application's UI, but cannot upload files. Therefore, it should be used exclusively as a root user - for the django /api/admin administrative panel.

To administer the user interface, log in to the http://your.host/signup application and then through the first superuser, grant admin rights to the registered user in the Django administration panel http://your.host. Thus a registered and authorized administrator user will be able to perform all actions through the administrative panel of the user interface, as well as upload files to their own storage.
Open your browser and go to your host's home page. If you have done everything correctly, the home page of the application.
Go through the registration process and get administrator rights as mentioned in the note above.

Name		Name	Last commit message	Last commit date
Latest commit History 75 Commits
authentication		authentication
cloud		cloud
djangoProject		djangoProject
nginx		nginx
tests		tests
.coveragerc		.coveragerc
.dockerignore		.dockerignore
.example.env		.example.env
.example.env.db		.example.env.db
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yml		docker-compose.yml
entrypoint.sh		entrypoint.sh
manage.py		manage.py
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml
pytest.ini		pytest.ini

Liaksej/liaksej-tiny-cloud-backend

Folders and files

Latest commit

History

Repository files navigation

Tiny Cloud

Topics:

Application structure

Frontend (liaksej-tiny-cloud-frontend)

The root directory of the application consists of the following files and directories:

Backend structure (liaksej-tiny-cloud-backend)

The root directory of the application consists of the following files and directories:

Deployment instructions

Deployment process

Software preparation:

Preparing environment variables:

Application Build:

Launching the app:

About

Resources

Stars

Watchers

Forks

Languages