Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ import mongoCompassImage from './images/mongo-compass.png';

<Authors frontMatter={frontMatter} />

<SourceCodeMovieApp />

## What is write-behind caching?

Imagine you've built a movie streaming app. You used MongoDB as your data store, and as you needed to scale you implemented caching using Redis. This allows you to drastically speed up reads. However, now you are experiencing slowness when writing to MongoDB.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ While this works, you end up with a fair amount of duplicated code. Plus, it's d

Once you decide to handle authentication at the API gateway layer, you must decide where to store sessions.

Imagine you're building an e-commerce application that uses MongoDB as the primary data store. You could store sessions in MongoDB, but think about how many times the application needs to hit MongoDB to retrieve session information. If you have millions of customers, you don't want to go to MongoDB for every single request made to the API.
Imagine you're building an e-commerce application that uses MongoDB/ any relational database as the primary data store. You could store sessions in primary database, but think about how many times the application needs to hit primary database to retrieve session information. If you have millions of customers, you don't want to go to database for every single request made to the API.

This is where Redis comes in.

Expand Down Expand Up @@ -60,7 +60,7 @@ Use a **Redis Enterprise Cluster** to get the benefit of linear scaling to ensur

<SourceCode />

## API gateway caching in a microservices application with Redis and MongoDB
## API gateway caching in a microservices application with Redis

What's nice about a microservice architecture is that each service is set up so it can scale independently. Now, seeing as how each service might require authentication, you likely want to obtain session information for most requests. Therefore, it makes sense to use the API gateway to cache and retrieve session information and to subsequently pass the information on to each service. Let's see how you might accomplish this.

Expand Down
109 changes: 37 additions & 72 deletions docs/howtos/solutions/microservices/caching/index-caching.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import cacheHitImage from './images/redis-cache-aside-cache-hit.png';
Have you ever been in a situation where your database queries are slowing down?
Query caching is the technique you need to speed database queries by using different caching methods while keeping costs down! Imagine that you built an e-commerce application. It started small but is growing fast. By now, you have an extensive product catalog and millions of customers.

Thats good for business, but a hardship for technology. Your queries to MongoDB are beginning to slow down, even though you already attempted to optimize them. Even though you can squeak out a little extra performance, it isnt enough to satisfy your customers.
That's good for business, but a hardship for technology. Your queries to primary database (MongoDB/ Postgressql) are beginning to slow down, even though you already attempted to optimize them. Even though you can squeak out a little extra performance, it isn't enough to satisfy your customers.

## Why you should use Redis for query caching

Expand Down Expand Up @@ -99,7 +99,7 @@ If you use **Redis Enterprise** and a database that uses a JDBC driver, you can

<SourceCode />

## Caching in a microservices application with Redis and MongoDB
## Caching in a microservices application with Redis and primary database (MongoDB/ Postgressql)

In our sample application, the products service publishes an API for filtering products. Here's what a call to the API looks like:

Expand All @@ -114,32 +114,27 @@ In our sample application, the products service publishes an API for filtering p

### Get products by filter response (cache miss)

```json {30}
```json {25}
{
"data": [
{
"_id": 11000,
"data": {
"id": 11000,
"price": 3995,
"productDisplayName": "Puma Men Slick 3HD Yellow Black Watches",
"variantName": "Slick 3HD Yellow",
"brandName": "Puma",
"ageGroup": "Adults-Men",
"gender": "Men",
"displayCategories": "Accessories",
"styleImages": {
"default": {
"imageURL": "http://host.docker.internal:8080/images/11000.jpg"
}
},
"productDescriptors": {
"description": {
"value": "Stylish and comfortable, this motor sport inspired wrist watch from puma is designed with a plastic case and ..."
}
}
},
"productId": 11000
"productId": "11000",
"price": 3995,
"productDisplayName": "Puma Men Slick 3HD Yellow Black Watches",
"variantName": "Slick 3HD Yellow",
"brandName": "Puma",
"ageGroup": "Adults-Men",
"gender": "Men",
"displayCategories": "Accessories",
"masterCategory_typeName": "Accessories",
"subCategory_typeName": "Watches",
"styleImages_default_imageURL": "http://host.docker.internal:8080/images/11000.jpg",
"productDescriptors_description_value": "<p style=\"text-align: justify;\">Stylish and comfortable, ...",
"createdOn": "2023-07-13T14:07:38.020Z",
"createdBy": "ADMIN",
"lastUpdatedOn": "2023-07-13T14:07:38.020Z",
"lastUpdatedBy": null,
"statusCode": 1
}
//...
],
Expand All @@ -160,66 +155,36 @@ In our sample application, the products service publishes an API for filtering p
}
```

### Implementing cache-aside with Redis and MongoDB
### Implementing cache-aside with Redis and primary database (MongoDB/ Postgressql)

The following code shows the function used to search for products in MongoDB:
The following code shows the function used to search for products in primary database:

```typescript title="server/src/services/products/src/service-impl.ts"
async function getProductsByFilter(productFilter: IProductFilter) {
const mongo = getMongodb();
const filter: Document = {
statusCode: {
$eq: DB_ROW_STATUS.ACTIVE,
},
async function getProductsByFilter(productFilter: Product) {
const prisma = getPrismaClient();

const whereQuery: Prisma.ProductWhereInput = {
statusCode: DB_ROW_STATUS.ACTIVE,
};

if (productFilter && productFilter.productDisplayName) {
filter['data.productDisplayName'] = {
$regex: productFilter.productDisplayName,
$options: 'i',
whereQuery.productDisplayName = {
contains: productFilter.productDisplayName,
mode: 'insensitive',
};
}

const projection: IProduct = {
productId: 1,
data: {
id: 1,
price: 1,
productDisplayName: 1,
variantName: 1,
brandName: 1,
ageGroup: 1,
gender: 1,
displayCategories: 1,
styleImages: {
default: {
imageURL: 1,
},
},
productDescriptors: {
description: {
value: 1,
},
},
},
};
const products: Product[] = await prisma.product.findMany({
where: whereQuery,
});

const limit = 100;
const sort = {};
const products = await mongo.find(
COLLECTIONS.PRODUCTS.collectionName,
filter,
projection,
limit,
sort,
);
return products;
}
```

If you're familiar with MongoDB, this code should be pretty straightforward. You simply make a call to MongoDB to find products based on a filter on the products `displayName` property. You also define a projection object to specify which properties to get out of MongoDB. You can set up multiple columns for better fuzzy searching, but we simplified it for the purposes of this tutorial.
You simply make a call to primary database (MongoDB/ Postgressql) to find products based on a filter on the product's `displayName` property. You can set up multiple columns for better fuzzy searching, but we simplified it for the purposes of this tutorial.

Using MongoDB directly without Redis works for a while, but eventually it slows down. That's why you might use Redis, to speed things up. The cache-aside pattern helps you balance performance with cost.
Using primary database directly without Redis works for a while, but eventually it slows down. That's why you might use Redis, to speed things up. The cache-aside pattern helps you balance performance with cost.

The basic decision tree for cache-aside is as follows.

Expand All @@ -228,7 +193,7 @@ When the frontend requests products:
1. Form a hash with the contents of the request (i.e., the search parameters).
1. Check Redis to see if a value exists for the hash.
1. Is there a cache hit? If data is found for the hash, it is returned; the process stops here.
1. Is there a cache miss? When data is not found, it is read out of MongoDB and subsequently stored in Redis prior to being returned.
1. Is there a cache miss? When data is not found, it is read out of primary database and subsequently stored in Redis prior to being returned.

Here’s the code used to implement the decision tree:

Expand Down Expand Up @@ -256,7 +221,7 @@ router.post(API.GET_PRODUCTS_BY_FILTER, async (req: Request, res: Response) => {
result.data = docArr;
result.isFromCache = true;
} else {
// get data from mongodb
// get data from primary database
const dbData = await getProductsByFilter(body); //method shown earlier

if (body && body.productDisplayName && dbData.length) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
The e-commerce microservices application discussed in the rest of this tutorial uses the following architecture:

1. `products service`: handles querying products from the database and returning them to the frontend
1. `orders service`: handles validating and creating orders
1. `order history service`: handles querying a customer's order history
1. `payments service`: handles processing orders for payment
1. `digital identity service`: handles storing digital identity and calculating identity score
1. `api gateway`: unifies services under a single endpoint
1. `mongodb`: serves as the primary database, storing orders, order history, products, etc.
1. `redis`: serves as the **stream processor** and caching database
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,11 @@ The e-commerce microservices application discussed in the rest of this tutorial
1. `payments service`: handles processing orders for payment
1. `digital identity service`: handles storing digital identity and calculating identity score
1. `api gateway`: unifies services under a single endpoint
1. `mongodb`: serves as the primary database, storing orders, order history, products, etc.
1. `redis`: serves as the stream processor and caching database
1. `mongodb/ postgresql`: serves as the primary database, storing orders, order history, products, etc.
1. `redis`: serves as the **stream processor** and caching database

:::info

You don't need to use MongoDB/ Postgresql as your primary database in the demo application; you can use other <u>[prisma supported databases](https://www.prisma.io/docs/reference/database-reference/supported-databases)</u> as well. This is just an example.

:::
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ You eventually land on the following architecture:
1. `order history service`: handles querying a customer's order history
1. `payments service`: handles processing orders for payment
1. `api gateway`: unifies the services under a single endpoint
1. `mongodb`: serves as the write-optimized database for storing orders, order history, products, etc.
1. `mongodb/ postgresql`: serves as the write-optimized database for storing orders, order history, products, etc.

:::info

You dont need to use MongoDB as your write-optimized database; you can use other databases such as a SQL database as well. This is just an example.
You don't need to use MongoDB/ Postgresql as your write-optimized database in the demo application; you can use other <u>[prisma supported databases](https://www.prisma.io/docs/reference/database-reference/supported-databases)</u> as well. This is just an example.

:::
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
The e-commerce microservices application consists of a frontend, built using [Next.js](https://nextjs.org/) with [TailwindCSS](https://tailwindcss.com/). The application backend uses [Node.js](https://nodejs.org). The data is stored in
[Redis](https://redis.com/try-free/) and MongoDB. Below you will find screenshots of the frontend of the e-commerce app:

- `Dashboard`: Shows the list of products with search functionality

![redis microservices e-commerce app frontend products page](images/design-dashboard.png)

- `Shopping Cart`: Add products to the cart, then check out using the "Buy Now" button
![redis microservices e-commerce app frontend shopping cart](images/design-cart-2.png)

- `Order history`: Once an order is placed, the `Orders` link in the top navigation bar shows the order status and history

![redis microservices e-commerce app frontend order history page](images/design-order-history.png)
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
The e-commerce microservices application consists of a frontend, built using [Next.js](https://nextjs.org/) with [TailwindCSS](https://tailwindcss.com/). The application backend uses [Node.js](https://nodejs.org). The data is stored in [MongoDB](https://www.mongodb.com/) and [Redis](https://redis.com/try-free/). Below you will find screenshots of the frontend of the e-commerce app:
The e-commerce microservices application consists of a frontend, built using [Next.js](https://nextjs.org/) with [TailwindCSS](https://tailwindcss.com/). The application backend uses [Node.js](https://nodejs.org). The data is stored in
[Redis](https://redis.com/try-free/) and MongoDB/ Postgressql using [Prisma](https://www.prisma.io/docs/reference/database-reference/supported-databases). Below you will find screenshots of the frontend of the e-commerce app:

- `Dashboard`: Shows the list of products with search functionality

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
:::tip GITHUB CODE

Below is a command to the clone the source code for the application used in this tutorial

git clone --branch v1.0.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions

:::
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

Below is a command to the clone the source code for the application used in this tutorial

git clone --branch v1.0.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions
git clone --branch v4.2.0 https://github.com/redis-developer/redis-microservices-ecommerce-solutions

:::
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading