- 1. What is Castive?
- 2. Using 2 Databases - Redis & MongoDB
- 3. Improved authentication system using 2 tokens and redis
- 4. Ratelimiting feature
- 5. Password reset and mail confirmation system
- 6. Improving response times and reducing the load on MongoDB
- 7. Partial and full-text search system
- 8. Protection for XSS & NoSQL Injection and Parameter pollution attacks
- 9. Highly Customizable responses using many available query parameters
- 10. Technologies used to build Castive
- LICENSE
Castive is a platform where people can create accounts, build and share playlists with their favourite movies and tv series. People can follow each other, like other people's playlists or block other users so that their playlists are not visible to those users.
This project is the backend service for Castive.
MongoDB was chosen over traditional SQL databases since it provides a handful of tools like full text search and documents structures are easily changed unlike a SQL database.
Redis is used alongside MongoDB to store jwt tokens of all kinds to improve response times and seperate the application logic from token system.
- Caching
Redis is also used for caching responses from TMDB and Announcements that are stored in the main (MongoDB) database. This allows for faster generated responses and reduces the number of requests sent to TMDB.
Most applications use only access token and some of those have the silent authentication system. However this system is not easy to implement and fairly complicated compared to refresh tokens. That's why I decided to go with access and refresh token system.
The improvement over other systems is that both tokens get stored in Redis database rather than main database. This allows the server to respond quicker to subsequent requests made with access tokens and refresh tokens since every request requires a validation of the tokens.
Implementation of this system can be seen in src/util/jwt.js
and src/controllers/v1/auth.controllers.js
.
Castive is using rate limiting in most routes to prevent brute force attacks and abuse. Implementation of rate limiting can be found in src/util/limiter.js
.
Just like every single social platform, Castive has its own mailing system to send password reset tokens and mail confirmations. Implementation can be found in src/util/mailer.js
* Tokens sent with all mails are also stored in Redis to validate afterwards and improve security.
-
Image System
For the image system, I've used a seperate Image model. The system works in the following order:- User uploads an image
- The image is then stored in a
Image
document withoriginal
property set - When the image's
small
ormedium
format is requested, they are generated on the fly for just one time. - For every other subsequent request the requested type is sent with response
That is, ifsmall
andmedium
are not requested, those fields will remainnull
, therby not becoming a burden on database. Moreover, having thesmall
andmedium
version of the images opens the possiblity of requesting those versions wheneveroriginal
is simply not needed, thereby improving response times. -
Virtual Fields in Models
There are simply no fields to store some of the properties of users and lists. For instance, under auser
document, onlyfollowing
property is stored. Wheneverfollowers
property is needed for a response, it gets generated using Mongoose's very handful.populate()
method. The implementation of populating field can be seen in bothsrc/models/user.model.js
andsrc/models/list.model.js
.
Thanks to MongoDB, it was easy to implement a full-text search without dealing with systems like Elastic Search. Implementing full text search is as simple as follows,
.find(
{
$text: {
$search: q,
$caseSensitive: false,
$diacriticSensitive: false,
},
_id: { $ne: id },
blocked: { $nin: [id] },
},
{ score: { $meta: 'textScore' } }
)
For the partial text search part, I use Regular Expressions whenever the result from full text search yields 0 results.
The full implementation can be seen in src/models/user.model.js
.
Security is important, especially if you are building a social platform. To overcome some common problems, I took help from some packages such as, Helmet, MongoSanitize and HPP.
Almost all of the endpoints take a great amount of query parameters to customize the response according to the needs. This allows the requester to exclude any information that is not needed and improve response times. An example can be seen below.
/users/me?following=1&followers=0&lists=true&blocked=false&library=1
Castive uses a number of technologies to operate.
- Express - Fast node.js network app framework
- node.js - Evented I/O for the backend
- JWT - Industry standard RFC 7519 method for representing claims securely between two parties
- MongoDB - A document database
- Redis - In-memory data structure store
- Mongoose - Elegant mongodb object modeling for NodeJS
- TMDB - The Movie Database (TMDB) is a community built movie and TV database.
* This product uses the TMDB API but is not endorsed or certified by TMDB.
This project is under GNU GPLv3 license. See LICENSE and COPYING for more.
For anything related to the project, contact me at ahmetberkegokmen@gmail.com.