This repository is for a project of the course Advanced System Programming.
WIKI IS NOT A FILE OR A FOLDER, IT'S ON THE GITHUB WIKI PAGE.
This project is done in steps through exercises and the end goal is to make a gmail clone.
https://github.com/Eranshc/Gmail-Project-ASP
The code of each exercise can be found on it's own branch (with the format: 'Exercise-{num}').
A Gmail clone, that uses a react frontend for the website, an express backend, together with a C++ server that acts as a bloom filter that stores blacklisted urls. This projects follows coding principles such as TDD, SOLID, MVC, and more. In this project, users can send mails, add/remove labels, edit mails and labels, add/remove urls from spam, view all mails received, sent, spam, all, and more. Users can mark a mail as important. Users can create drafts that can be later published.
Go to the project's root directory and run: sudo docker-compose build
Then run: sudo docker-compose up -d react_app
The website will be available on: http://localhost:3000
To then close the servers, use sudo docker-compose down
To run only the js backend server run:
sudo docker-compose up -d js_server
This will start the server at port 8080.
In order to run the python client for the bloom filter run:
sudo docker-compose run client cpp_server <port>
In order to only run the bloom-filter server run:
sudo docker-compose up -d cpp_server
In order to run a test, run the command: sudo docker-compose run tests ./<test>
By default the test will be Server_TDD.
If the test is Server_TDD (which expects input from a client), run:
sudo docker-compose up -d tests
Now in order to connect the client to a test if needed just run:
sudo docker-compose run client tests <port>
To see the output for this test and in general if using docker-compose up you can run:
sudo docker-compose logs -f
Tests that currently exist:
- Delete_TDD
- CommandOutput_TDD
- Server_TDD
- Persistency_TDD
- Blacklist_TDD
- CheckBlacklist_TDD
- Configuration_TDD
Note: Running sudo docker-compose run tests ./Server_TDD will not allow connections to the server resulting in a non-working test.
Create a folder named build, go into it and run inside of it cmake .. && make
All the relavent bloom-filter executables will appear there.
Now go to js_server folder and run node app.js
This will conenct the backend of the mails system to the bloom filter.
To run the js api server, first make sure the bloom-filter server is running. The server will run without it, but all access to /api/blacklist and attempts at creating mails with links will result in failure.
Do make sure to go to the .env file in js_server/config and change the blacklist host to localhost instead of cpp_server.
From there, just go to the /src/js_server directory and run node app.js
(after running npm i).
Everything should be working by now, and the api server will run at port 8080.
To run the website itself go to the frontend_react_server in src and run npm start (after running npm i), which will open the website on port 3000.
The blacklist server expects input line by line from standard input:
-
Command-Line Arguments (Configuration):
<port> <size> <hash_specifier_1> <hash_specifier_2> ...<port>: The port that the server will listen to<size>: The number of bits in the Bloom filter's bit array (integer).<hash_specifier_n>: Integers specifying which hash functions to use. The interpretation depends on the implementation that follows the assignment examples so :1might meanstd::hash,2might mean a second distinct hash function orstd::hashapplied twice, based on the SOLID design.- Example:
5555 8 1 2means an 8-bit array using two hash strategies defined as '1' and '2', while the server listens on port 5555. - Example:
5555 8 1means an 8-bit array using one hash strategy defined as '1', while the server listens on port 5555.
-
Subsequent Lines For Bloom-Filter Server (Commands):
POST <URL>: Add the<URL>to the blacklist. The Bloom filter and the persistent blacklist file are updated.GET <URL>: Check if<URL>is potentially blacklisted.DELETE <URL>: Delete<URL>from the blacklist (but not it's matching bits from the bit array)- Any other format or lines with incorrect structure for these commands will be fail, and the program will wait for a valid input.
- For command
GET <URL>:- If the Bloom filter indicates the URL is not present, output: false
- If the Bloom filter indicates the URL might be present:
- If the URL is found in the actual persistent blacklist, output: true true
- If the URL is not found in the actual persistent blacklist (a false positive), output: true false
The application maintains persistence through two files stored in the data/ directory.
The system employs a flexible hashing strategy based on the configuration provided in the first input line. It uses dependency injection or a strategy pattern to decouple the BloomFilter class from specific hash function implementations. This adheres to SOLID principles, allowing easy modification or addition of hash functions in the future without altering the core filter logic.
Go to the src/frontend_react_server (after running all other servers, or just use docker-compose as instructed above) and run npm start. This will start the react server at port 3000.
Then you can open your browser and navigate to http://localhost:3000 to see the react app in action.
Once there you will see the login page
You should press 'Sign up' to proceed to the sign-up page.
Here enter your desired username, email adress, password and your profile picture, and press 'Create Account' that will register you in the system.
Now press 'Login' to proceed to the login page.
Enter your username and password and press 'Login' to login and see the main page.
Repeat this process to create two users. To use two users from the same computer, either open two different types of browser, or open one anonymous session of a browser.
Press 'compose' and then you will see the compose page. Enter the email of the user you want to send a mail to, enter the subject and the content of the mail.
You can shrink the page or close the mail, closing the mail after it's ready to be sent will not send the mail but make a draft of it.
Press 'Submit' in order to send the email.
Now press 'Sent' to see the mail that you sent.
Pressing the mail will reveal its content.
Now you can shrink or close the mail if you want.
You can also delete the mail by pressing the bin button that will delete it on both participetns.
You can edit the mail via pressing the pen button that will open the compose page with the mail's content.
Pressing the '+' button will allow you to add a mail label.
You can add as many labels as you want and edit existing ones by right clicking on them and pressing 'EDIT' or 'DELETE' to delete one.
Open the mail again and press 'Assign label' to assign a label to the mail.
You can also unassign a label by cliking on 'Unassign label' and pick the one you want to unassign.
Now cliking on the label will show you all the mails that have that label.
Press it again to go back to the main page.
You can press the the important icon '!' to mark a mail as important.
This will highlight the mail and make it stand out but also make it appear in the important section.
You can also unmark it by pressing the same icon again.
Not every url is safe to click on, so we have a blacklist of urls that we will not allow.
The user may add a url to this list by 'BlackList' button.
If the user send a mail with a url that is in the blacklist, the mail will go the spam of the receiver.
Sometimes we close the compose window by accident and as such we need to save what we wrote as a draft.
The user may close the compose window without fear of losing the mail he had written but not sent yet simply by pressing the 'x' button.
Then the app will create a new draft mail automatically for us.
And by pressing the Publish button the mail is sent to the intended user as is.
Of course, modifying is always an option.
Close the docker with 'sudo docker compose down'.
An exemple of the single python client and bloom-filter server:
Now the files have changed:
Example of multithreading:
Example of api run:
Second request doesn't show error because of a visual problem in the CLI, should say: {"error":"Missing fields: name, email, password or image"}
The function names didn't really affect our code, as all we had to change was the vector of the commands to a map of strings and commands. What helped us a lot out here was the fact we had an entire file dedicated to commands in general, which allowed us to easily initialize whichever struct we needed, as well as easily modify it with little to no changes.
Same exact thing goes with the new command - the delete command. all we had to do is add an instance of it to the map of commands.
The change in the output of commands made a little bit of difference but all that we needed to do was add about one line in each command in order for it to give the new output.
With the output it was a little bit different, yet still - most of the changes weren't in the existing code. The thing that helped us this time was creating an i/o handler interface which alowed us to switch pretty easily to socket i/o's with the new outputs.
This very same thing helped us with the inputs as well - all we had to change in the pre-existing code was to call the socket i/o handler, and bam! Almost as if someone(ahem) hinted out that where we receive our inputs from might change!
For each request that requires a user, a header called "Authorization" is required. The Authorization header follows the format of "Bearer |token|" where token is a JWT token that contains the user-id of the user creating the request. The requests that require this header are everything in the /api/mails and /api/labels routes. Every request for those routes, requires a user. Editing an email is only possible with the sender's id, the receiver can't edit the mail. Each request satisfies that API requirements in the 3rd exercise instructions.
Mails fields are: title, content, sender, receiver, is_draft, is_spam, is_seen, is_important. Sender and receiver are user id's, everything apart from sender is received in the body. sender is received through the JWT token.
Labels fields are: name, user-id, mails. Name is received in the body, user-id is received through the JWT token. mails are received through the /labels/:id route with POST with a mail_id.
User fields are: name, email, password, profilePicture, mails. name, email, and password are received in the body, mails starts as an empty array and is filled with the user's sent and received mails. The profilePicture is received through req.files (This request assumes multipart content type) All images are uploaded to a designated directory after upload. The profilePicture field in the user json consists of metadata about the image, such as it's name, mimetype, path (that can be served statically) and more. The images are uploaded to the directory specified in .env file as PROFILE_PIC_DIR. Images uploaded are compressed and resized to 300x300.
Tokens fields are: name, password. They are passed through the body. The response of the POST request is a JWT token containing the user-id of the user matching the name and password given.
Blacklist fields are: url. When posting a url it is passed through the body, and when deleting passed as a url parameter.
The JWT key should be inside a /src/js_server/config directory. Inside this directory, in a file called .env, there is an environment variable called JWT_SECRET. This environment variable is the key of the tokens.