diff --git a/docs/create/aws/analytics-using-aws/analytics3.png b/docs/create/aws/analytics-using-aws/analytics3.png new file mode 100644 index 0000000000..9d4872947e Binary files /dev/null and b/docs/create/aws/analytics-using-aws/analytics3.png differ diff --git a/docs/create/aws/analytics-using-aws/analytics4.png b/docs/create/aws/analytics-using-aws/analytics4.png new file mode 100644 index 0000000000..cf3dbb13de Binary files /dev/null and b/docs/create/aws/analytics-using-aws/analytics4.png differ diff --git a/docs/create/aws/analytics-using-aws/analytics_dashboard.png b/docs/create/aws/analytics-using-aws/analytics_dashboard.png new file mode 100644 index 0000000000..76774c0b5d Binary files /dev/null and b/docs/create/aws/analytics-using-aws/analytics_dashboard.png differ diff --git a/docs/create/aws/analytics-using-aws/analytics_traffic.png b/docs/create/aws/analytics-using-aws/analytics_traffic.png new file mode 100644 index 0000000000..e1fc2caa7b Binary files /dev/null and b/docs/create/aws/analytics-using-aws/analytics_traffic.png differ diff --git a/docs/create/aws/analytics-using-aws/index-analytics-using-aws.mdx b/docs/create/aws/analytics-using-aws/index-analytics-using-aws.mdx new file mode 100644 index 0000000000..8eb5faf6b6 --- /dev/null +++ b/docs/create/aws/analytics-using-aws/index-analytics-using-aws.mdx @@ -0,0 +1,352 @@ +--- +id: index-analytics-using-aws +title: How to Build and Deploy Your Own Analytics Dashboard on AWS using NodeJS and Redis +sidebar_label: Building Analytics Dashboard on using NodeJS and then deploy it on AWS +slug: /create/aws/analytics-using-aws +authors: [ajeet] +--- + +An interactive analytics dashboard serves several purposes. They allow you to share data and provide you with all those vital information to make game-changing decisions at a faster pace. Building a real-time dynamic dashboard using a traditional relational database might require a complex set of queries. By using a NoSQL database like Redis, you can build a powerful interactive and dynamic dashboard with a small number of Redis commands. + +Let’s take a look at how this was achieved. + + +- What will you build? +- What will you need? +- Getting started +- How does it work? +- How data is stored +- Navigating the application + + +### 1. What will you build? + +You’ll build an analytics dashboard app that uses Redis Bitmap written in NodeJS (JavaScript) and then deploy it to AWS + +Ready to get started? Ok, let’s dive straight in. + + +### 2. What will you need? + +- [NodeJS](https://developer.redis.com/develop/node): used as an open-source, cross-platform, backend JavaScript runtime environment that executes Javascript code outside a web browser. +- [Redis Enterprise Cloud](https://developer.redis.com/create/rediscloud): used as a real-time database, cache, and message broker.i +- [NPM](https://www.npmjs.com/): used as a package manager. It allows you to build node apps. + +### 3. Getting Started + +### Prepare the environment + +- Install Node - v12.19.0 +- Install NPM - v6.14.8 + + +### Step 1. Sign up for a Free Redis Enterprise Cloud Account + +[Follow this tutorial](https://developer.redis.com/create/aws/redis-on-aws) to sign up for a free Redis Enterprise Cloud account. + +![image](analytics3.png) + +Choose AWS as a Cloud vendor while creating your new subscription. At the end of the database creation process, you will get a Redis Enterprise CLoud database endpoint and password. You can save it for later use. + +![image](analytics4.png) + + + + +### Step 2. Clone the repository + + ```bash + git clone https://github.com/redis-developer/basic-analytics-dashboard-redis-bitmaps-nodejs + ``` + + +### Step 3. Setting up backend + + +First we will be settin up environmental variables + +Go to /server folder (cd ./server) and then execute the below command: + + ```bash + cp .env.example .env + ``` + +Open .env file and add Redis Enterprise Cloud Database Endpoint URL, port and password as shown below: + +``` + +PORT=3000 + +# Host and a port. Can be with `redis://` or without. +# Host and a port encoded in redis uri take precedence over other environment variable. +# preferable +REDIS_ENDPOINT_URI=redis://redis-XXXX.c212.ap-south-1-1.ec2.cloud.redislabs.com:15564 + +# Or you can set it here (ie. for docker development) +REDIS_HOST=redis-XXXX.c212.ap-south-1-1.ec2.cloud.redislabs.com +REDIS_PORT=XXXX + +# You can set password here +REDIS_PASSWORD=reXXX + +COMPOSE_PROJECT_NAME=redis-analytics-bitmaps +``` + + +### Step 4. Install dependencies + + ```bash + npm install + ``` + + +### Step 5. Run the backend + + ```bash + npm run dev + ``` + +### Step 6. Setting up the frontend + +Go to /client folder (cd ./client) and then: + + ```bash + cp .env.example .env + ``` + +Add the exact URL path and port number of your choice for VUE_APP_API_URL parameter as shown below: + +``` +VUE_APP_API_URL=http://localhost:3000 +``` + + +### Step 7. Install dependencies + + ```bash + npm install + ``` + +### Step 8. Run the frontend + + ```bash + npm run serve + ``` + +![analytics](analytics_traffic.png) + +### How does it work? + +#### How the data is stored: + +The event data is stored in various keys and various data types. + +For each of time spans: + +- year: like 2021 +- month: like 2021-03 (means March of 2021) +- day: like 2021-03-03 (means 3rd March of 2021) +- weekOfMonth: like 2021-03/4 (means 4th week of March 2021) +- anytime + +and for each of scopes: + +- source +- action +- source + action +- action + page +- userId + action +- global + +and for each of data types (types): + +- count (Integer stored as String) +- bitmap +- set + +Is generated key like: + + + ```bash + rab:{type}[:custom:{customName}][:user:{userId}][:source:{source}][:action:{action}][:page:{page}]:timeSpan:{timeSpan} + ``` + +where values in [] are optional. + +- For each generated key like rab:count:*, data is stored like: INCR {key} +Example: + + ```bash + INCR rab:count:action:addToCart:timeSpan:2015-12/3 + ``` +- For each generated key like: rab:set:*, data is stored like: SADD {key} {userId} +Example: + + ```bash + SADD rab:set:action:addToCart:timeSpan:2015-12/3 8 + ``` +- For each generated key like rab:bitmap:*, data is stored like: SETBIT {key} {userId} 1. +Example: + + ```bash + SETBIT rab:bitmap:action:addToCart:timeSpan:2015-12/3 8 1 + ``` +### Cohort data + +- We store users who register and then bought some products (action order matters). +- For each buy action in December we check if user performed register action before (register counter must be greater than zero). +- If so, we set user bit to 1 like: SETBIT rab:bitmap:custom:cohort-buy:timeSpan:{timeSpan} {userId} 1 + +- E.g User Id 2 bought 2 products on 2015-12-17. It won't be stored. +- E.g User Id 10 bought 1 product on 2015-12-17 and registered on 2015-12-16. + It will be stored like: SETBIT rab:bitmap:custom:cohort-buy:timeSpan:2015-12 10 1. +- We assume that user cannot buy without register. + +#### Retention data + +- Retention means users who bought on two different dates +- For each buy action we check if user bought more products anytime than bought on particular day (current purchase not included). +- If so, we add user id to set like: SADD rab:set:custom:retention-buy:timeSpan:{timeSpan} {userId} +- E.g User Id 5 bought 3 products on 2015-12-15. His retention won't be stored (products bought on particular day: 2, products bought anytime: 0). +- E.g User Id 3 bought 1 product on 2015-12-15 and before - 1 product on 2015-12-13. His retention will be stored (products bought on particular day: 0, products bought anytime: 1) like: SADD rab:set:custom:retention-buy:timeSpan:2015-12 3. + +#### How the data is accessed: + +- Total Traffic: + +December: ```BITCOUNT rab:bitmap:custom:global:timeSpan:2015-12``` +X week of December: ```BITCOUNT rab:bitmap:custom:global:timeSpan:2015-12/{X}``` +Example: + + ```bash + BITCOUNT rab:bitmap:custom:global:timeSpan:2015-12/3 + ``` + +- Traffic per Page ({page} is one of: homepage, product1, product2, product3): + +``` +December: BITCOUNT rab:bitmap:action:visit:page:{page}:timeSpan:2015-12 +``` + +Example: + + ```bash + BITCOUNT rab:bitmap:action:visit:page:homepage:timeSpan:2015-12 + ``` + +- X week of December: + +``` +BITCOUNT rab:bitmap:action:visit:page:{page}:timeSpan:2015-12/{X} +``` + +Example: + + ```bash + BITCOUNT rab:bitmap:action:visit:page:product1:timeSpan:2015-12/2 + ``` + +- Traffic per Source ({source} is one of: google, Facebook, email, direct, referral, none): + +December: + +``` +BITCOUNT rab:bitmap:source:{source}:timeSpan:2015-12 +``` + +Example: + + ```bash + BITCOUNT rab:bitmap:source:referral:timeSpan:2015-12 + ``` + +- X week of December: ```BITCOUNT rab:bitmap:source:{source}:timeSpan:2015-12/{X}``` + +Example: + + ```bash + BITCOUNT rab:bitmap:source:google:timeSpan:2015-12/1 + ``` + +- Trend traffic ({page} is one of: homepage, product1, product2, product3): + +- December: from ```BITCOUNT rab:bitmap:action:visit:{page}:timeSpan:2015-12-01``` to ```BITCOUNT rab:bitmap:action:visit:{page}:timeSpan:2015-12-31``` +- 1 Week of December: Similar as above, but from 2015-12-01 to 2015-12-07 +- 2 Week of December: Similar as above, but from 2015-12-08 to 2015-12-14 +- 3 Week of December: Similar as above, but from 2015-12-15 to 2015-12-21 +- 4 Week of December: Similar as above, but from 2015-12-22 to 2015-12-28 +- 5 Week of December: Similar as above, but from 2015-12-29 to 2015-12-31 +- Example: + + ```bash + BITCOUNT rab:bitmap:action:visit:homepage:timeSpan:2015-12-29 => BITCOUNT rab:bitmap:action:visit:homepage:timeSpan:2015-12-30 => BITCOUNT rab:bitmap:action:visit:homepage:timeSpan:2015-12-31 + ``` + +- Total products bought: + +- December: ```GET rab:count:action:buy:timeSpan:2015-12``` +- X week of December: ```GET rab:count:action:buy:timeSpan:2015-12/{X}``` +Example: + + ```bash + GET rab:count:action:buy:timeSpan:2015-12/1 + ``` + +- Total products added to cart: + +December: ```GET rab:count:action:addToCart:timeSpan:2015-12``` +X week of December: ```GET rab:count:action:addToCart:timeSpan:2015-12/{X}``` +Example: + + ```bash + GET rab:count:action:addToCart:timeSpan:2015-12/1 + ``` + +- Shares of products bought ({productPage} is on of product1, product2, product3): + +December: ```GET rab:count:action:buy:page:{productPage}:timeSpan:2015-12``` +Example: + + ```bash + GET rab:count:action:buy:page:product3:timeSpan:2015-12 + ``` + +- X week of December: ```GET rab:count:action:buy:page:{productPage}:timeSpan:2015-12/{X}``` +Example: + + ```bash + GET rab:count:action:buy:page:product1:timeSpan:2015-12/2 + ``` + +#### Customer and Cohort Analysis + +- People who registered: BITCOUNT rab:bitmap:action:register:timeSpan:2015-12 +- People who register then bought (order matters): BITCOUNT rab:bitmap:custom:cohort-buy:timeSpan:2015-12 +- Dropoff: (People who register then bought / People who register) * 100 [%] +- Customers who bought only specified product ({productPage} is one of: product1, product2, product3): + +``` +SMEMBERS rab:set:action:buy:page:{productPage}:timeSpan:2015-12 +``` + +Example: + + ```bash + SMEMBERS rab:set:action:buy:page:product2:timeSpan:2015-12 + ``` + +- Customers who bought Product1 and Product2: + +``` +SINTER rab:set:action:buy:page:product1:timeSpan:anytime rab:set:action:buy:page:product2:timeSpan:anytime +``` + +- Customer Retention (customers who bought on the different dates): SMEMBERS rab:set:custom:retention-buy:timeSpan:anytime + + +### References + +- [Project Source Code](https://github.com/redis-developer/basic-analytics-dashboard-redis-bitmaps-nodejs) +- [Use cases of Bitmaps](https://redis.io/topics/data-types-intro) +- [How to Build a Slack Bot to Retrieve Lost Files Using AWS S3 and RediSearch](/create/aws/slackbot) +- [How to Deploy and Manage Redis Database on AWS Using Terraform](/create/aws/terraform) diff --git a/docs/create/aws/bidding-on-aws/images/architecture.png b/docs/create/aws/bidding-on-aws/images/architecture.png new file mode 100644 index 0000000000..9070f493c0 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/architecture.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_10.png b/docs/create/aws/bidding-on-aws/images/image_10.png new file mode 100644 index 0000000000..8c5ff50924 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_10.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_11.png b/docs/create/aws/bidding-on-aws/images/image_11.png new file mode 100644 index 0000000000..b098307225 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_11.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_12.png b/docs/create/aws/bidding-on-aws/images/image_12.png new file mode 100644 index 0000000000..feb236bf99 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_12.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_13.png b/docs/create/aws/bidding-on-aws/images/image_13.png new file mode 100644 index 0000000000..3790cc29e0 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_13.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_2.png b/docs/create/aws/bidding-on-aws/images/image_2.png new file mode 100644 index 0000000000..31bc689f1b Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_2.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_3.png b/docs/create/aws/bidding-on-aws/images/image_3.png new file mode 100644 index 0000000000..0f3e2969de Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_3.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_4.png b/docs/create/aws/bidding-on-aws/images/image_4.png new file mode 100644 index 0000000000..4b20a07184 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_4.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_5.png b/docs/create/aws/bidding-on-aws/images/image_5.png new file mode 100644 index 0000000000..765296b166 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_5.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_6.png b/docs/create/aws/bidding-on-aws/images/image_6.png new file mode 100644 index 0000000000..3790cc29e0 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_6.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_7.png b/docs/create/aws/bidding-on-aws/images/image_7.png new file mode 100644 index 0000000000..8c5ff50924 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_7.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_8.png b/docs/create/aws/bidding-on-aws/images/image_8.png new file mode 100644 index 0000000000..765296b166 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_8.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_9.png b/docs/create/aws/bidding-on-aws/images/image_9.png new file mode 100644 index 0000000000..0f3e2969de Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_9.png differ diff --git a/docs/create/aws/bidding-on-aws/images/image_featured.png b/docs/create/aws/bidding-on-aws/images/image_featured.png new file mode 100644 index 0000000000..d9b3c241fb Binary files /dev/null and b/docs/create/aws/bidding-on-aws/images/image_featured.png differ diff --git a/docs/create/aws/bidding-on-aws/index-bidding-on-aws.mdx b/docs/create/aws/bidding-on-aws/index-bidding-on-aws.mdx new file mode 100644 index 0000000000..811b3369f1 --- /dev/null +++ b/docs/create/aws/bidding-on-aws/index-bidding-on-aws.mdx @@ -0,0 +1,301 @@ +--- +id: index-bidding-on-aws +title: How to Build a Real-Time Bidding Platform on AWS using NodeJS, AWS Lambda, and Redis +sidebar_label: Building a Real-Time Bidding Platform on AWS using NodeJS, AWS Lambda, and Redis +slug: /create/aws/bidding-on-aws +authors: [ajeet] +--- + +Digital technology has propelled us forward to an exciting new era and has transformed almost every aspect of life. We’re more interconnected than ever as communication has become instant. Working from home has now become the norm, helping us pivot to a new way of working during the pandemic. And our ability to reduce carbon emissions by attending work-related events online has meant that we’ve taken greater strides to combat global warming. Continuing this trend is [Shamshir Anees and his team](https://github.com/shamshiranees), who have created an application that can host digital auctions. By using Redis, data transmission between components was carried out with maximum efficiency, providing users with real-time bidding updates on the dashboard. + +Let’s take a look at how this was achieved. We’d also like to point out that we have a diverse range of exciting applications for you to check out on the [Redis Launchpad](https://launchpad.redis.com). + +- What will you build? +- What will you need? +- Architecture +- How does it work? +- Getting started +- How data is stored +- Navigating the application + +### What will you build? + +You’ll build an application that will allow users to attend and take part in digital auctions. The application will allow users to create an account, put in bids, and even set up their own auction. Below we’ll uncover the required components, their functionality, and how to deploy them within this architecture. + +Ready to get started? Ok, let’s dive straight in. + +### What will you need? + +- [NodeJS](https://developer.redis.com/develop/node): used as an open-source, cross-platform, backend JavaScript runtime environment that executes Javascript code outside a web browser. +- [Amazon Cognito](https://aws.amazon.com/cognito/): used to securely manage and synchronize app data for users on mobile. +- [Redis Enterprise Cloud](https://developer.redis.com/create/rediscloud): used as a real-time database, cache, and message broker. +- [RedisJSON](https://developer.redis.com/howtos/redisjson/getting-started): used to store, update and fetch JSON values from Redis. +- [Socket.IO](https://socket.io/): used as a library that provides real-time, bi-directional, and event-based communication between the browser and the server. +- [AWS Lambda](https://aws.amazon.com/lambda/): used as a serverless compute service that runs your code in response events and manages the underlying compute service automatically for you. +- [Amazon SNS/Amazon SES](https://aws.amazon.com/sns/): a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication. + +### Architecture + +![My Image](images/architecture.png) + +### How does it work? + +#### All auctions + +NodeJS connects to the Redis Enterprise Cloud database. + +The frontend then communicates with the NodeJS backend through API calls. + +```GET : /api/auctions``` fetches all the keys from Auctions Hash. + +NodeJS uses the Redis module to work with Redis Enterprise Cloud. The Redis client is then created using the Redis credentials and hmget(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +#### Each auction + +```GET : /api/auctions/{auctionId}``` fetches each auction item by id. + +NodeJS uses the Redis module to work with Redis Cloud. The Redis client is then created using the Redis credentials and hmget(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +All bidding data of an auction item + +```GET : /api/bidding/{auctionId}``` + +NodeJS uses the Redis module to work with Redis Cloud. The Redis client is then created using the Redis credentials and hmget(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +#### Profile settings + +```GET : /api/settings``` + +NodeJS uses the Redis module to work with Redis Cloud. The Redis client is then created using the Redis credentials and hmget(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +#### User info + +```GET : /api/users/{email}``` + +NodeJS uses the Redis module to work with Redis Cloud. The Redis client is then created using the Redis credentials and hmget(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +### Getting started + +### Prerequisites + +- [NodeJS](https://nodejs.org/en/) +- [NPM](https://www.npmjs.com/) + +### Step 1. Sign up for a Free Redis Enterprise Cloud Account + +[Follow this tutorial](https://developer.redis.com/create/aws/redis-on-aws) to sign up for a free Redis Enterprise Cloud account. + +![image](sign3.png) + +Choose AWS as a Cloud vendor while creating your new subscription. At the end of the database creation process, you will get a Redis Enterprise CLoud database endpoint and password. You can save it for later use. + +![image](sign4.png) + + +### Step 2: Clone the backend GitHub repository + +``` +https://github.com/redis-developer/NR-digital-auction-backend +``` + +### Step 3. Install the package dependencies + +The 'npm install' is a npm cli-command that does the predefined thing i.e install dependencies specified inside package.json + +``` +npm install +``` + +### Step 4. Setting up environmental variable + + + +``` +export REDIS_END_POINT=XXXX +export REDIS_PASSWORD=XXX +export REDIS_PORT=XXX +``` + +### Step 5. Building the application + +``` +npm run build +``` + +### Step 6. Starting the application + +``` +npm start +``` + +### Step 7. Cloning the Frontend GITHUB repository + +``` +git clone https://github.com/redis-developer/NR-digital-auction-frontend +``` + +### Step 8. Building the application + +``` +npm run build +``` + +### Step 9. Starting the application + +``` +npm start +``` + +### Step 10. Accessing the application + +![accessing](images/image_2.png) + +### Step 11. Signing up to the application + +![signing](images/image_3.png) + +### Step 12. Sign-in + +![sign](images/image_4.png) + + +### Step 13. Accessing the dashboard + +![access](images/image_5.png) + +### Step 14. Listing the auction item + +![Listing](images/image_6.png) + + +### Step 15. Accessing the bidding page + +![accessing](images/image_7.png) + + +### How data is stored + +The Redis Enterprise Cloud Database with RedisJSON module is what you’ll use to install the data. + +### Auctions + +- Type - Redis Hash. +- Used for storing auctions data. +- UUID generated from the backend (NodeJS) serves as the key +- JSON data which represents the Auction object and includes the following keys + - auctionId + - auctionItemName + - description + - lotNo + - quantity + - buyersPremium + - itemUnit + - minBidAmount + - bidIncrement + - startDateTime + - endDateTime + - images + - currentBid +- NodeJS connects to the Redis Cloud database. The Frontend communicates with the NodeJS backend through API calls. +- POST : /api/auctions. +- The request body has JSON data to be inserted into the database. +- NodeJS uses the Redis module to work with Redis Cloud. The Redis client is created. using the Redis credentials and hmset(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +### Biddings + +- Type - Redis Hash +- Used for storing the bids placed on each auction item +- NodeJS connects to the Redis Cloud database. The Frontend communicates with the NodeJS backend through API calls. +- POST : ```/api/bidding``` +- The request body has JSON data to be inserted into the database. +- AuctionId from request body serves as the key +- JSON data which includes keys: + + - currentBid + - currentBidTime + - currentBidEndTime, and + - biddings array (id, auctionId, userId, username, bidAmount, bidTime) + +- The bidding array has all of the bids placed for a particular auction item. +- Based on the current BidEndTime and BidTime, the auction end date is extended based on the Dynamic closing concept. +- Current dynamic closing logic - If a new bid is placed within the last 5 minutes of the auction end time, the end time is extended by 1 hour. +- This will be configurable in the SaaS solution. +- NodeJS uses the Redis module to work with Redis Cloud. The Redis client is created using the Redis credentials and hmset(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +### Profile Settings + +- Type - string +- JSON data which includes keys - serves as a value +- NodeJS connects to the Redis Cloud database. The frontend communicates with the NodeJS backend through API calls. +- POST : ```/api/settings``` +- The request body has JSON data to be inserted into the database. +- NodeJS uses the Redis module to work with Redis Cloud. The Redis client is created using the Redis credentials and hmset(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +### Users + +- Type - Redis Hash +- Used for storing user details +- NodeJS connects to the Redis Cloud database. The Frontend communicates with the NodeJS backend through API calls. +- POST : ```/api/users``` +- The request body has JSON data to be inserted into the database +- The email id serves as the key +- The JSON data which includes keys - serves as a value +- NodeJS uses the Redis module to work with Redis Cloud. The Redis client is created using the Redis credentials and hmset(). This is the equivalent of the HMSET command that’s used to push data to the Redis database. + +### Navigating the application + +### Creating an account + +When you go onto the Digital Auction’s homepage, you’ll come across a range of items that are to be auctioned (see below). Click on the ‘Welcome’ button to create an account. + +![creating](images/image_8.png) + +You’ll then be taken to the sign-up page. Enter your details and click ‘sign-up.’ Once you’ve completed the sign-up form, you’ll receive a confirmation email to activate your account. + +### Placing a bid + +Go to the homepage to have access to view all of the items and their auction details. All of the data here is being populated by RedisJSON and Redis Cloud. Scroll through the page and click on the item that you want to place a bid for. + +![placing](images/image_9.png) + +When you click on an item, you’ll see the details for the bidding process at the top of the page. You’ll also have the option to set a reminder by receiving an email of whenever the bidding process of this item begins. + +On the right-hand side of the image, you’ll see the highest bid that’s been placed for this item. Below is a list of previous bids made by different users which are updated in real-time. + +Click on the ‘Place Bid’ button to make a bid. + +To access the meta-data information or view more images of the item, simply scroll down the page (see below). + +![placebid](images/image_10.png) + +### Viewing your Bidding History + +Click on ‘My biddings’ at the top of the navigation bar to view your bidding history (see below). + +![view](images/image_11.png) + +### Viewing upcoming auctions + +Click on ‘Auctions’ at the top of the navigation bar to view all upcoming auctions. + +![auction](images/image_12.png) + +### Conclusion: Leveraging Redis and AWS to Empower Auctioneers with Real-time Data + +Digital technology has had a ripple effect across all aspects of modern life. The ability to complete important tasks online instead of in-person has revolutionized the way we live, helping us to reduce carbon emissions, save time from traveling and have instant access to reams worth of data that we never had before. + +However, the success of such events hinges on a database’s ability to transmit data in real-time. Any blips in transmission would create a disconnect between users and the auction, impeding auctioneers’ reactions to bids. This would only result in frustration, disengagement, and a complete divorce of users from the application. + +But thanks to Redis, the components that made up the architecture system became vastly more interconnected so data was able to be sent, processed, and received in real-time. Achieving this paves the way for a smooth bidding process where users can interact with events in real-time without interruptions, ultimately enhancing the functionality of the app. + +[NR-Digital-Auction](https://launchpad.redis.com/?id=project%3ANR-digital-auction-frontend) is a fantastic example of how innovations can be brought to life by using Redis. Everyday programmers are experimenting with Redis to build applications that are impacting everyday life from around the world and you can too! + +So what can you build with Redis? For more inspiration, you can head over to the Redis Launchpad to access an exciting range of applications. + +Don't miss out on your $200 credit. Plus, your opportunity win a Tesla! + + +
+ Redis Try Free +
+ diff --git a/docs/create/aws/bidding-on-aws/matrix200.png b/docs/create/aws/bidding-on-aws/matrix200.png new file mode 100644 index 0000000000..0465899283 Binary files /dev/null and b/docs/create/aws/bidding-on-aws/matrix200.png differ diff --git a/docs/create/aws/bidding-on-aws/sign3.png b/docs/create/aws/bidding-on-aws/sign3.png new file mode 100644 index 0000000000..9d4872947e Binary files /dev/null and b/docs/create/aws/bidding-on-aws/sign3.png differ diff --git a/docs/create/aws/bidding-on-aws/sign4.png b/docs/create/aws/bidding-on-aws/sign4.png new file mode 100644 index 0000000000..cf3dbb13de Binary files /dev/null and b/docs/create/aws/bidding-on-aws/sign4.png differ diff --git a/docs/create/aws/chatapp/chatapp2.png b/docs/create/aws/chatapp/chatapp2.png new file mode 100644 index 0000000000..26d8e55ab6 Binary files /dev/null and b/docs/create/aws/chatapp/chatapp2.png differ diff --git a/docs/create/aws/chatapp/chatapp3.png b/docs/create/aws/chatapp/chatapp3.png new file mode 100644 index 0000000000..9d4872947e Binary files /dev/null and b/docs/create/aws/chatapp/chatapp3.png differ diff --git a/docs/create/aws/chatapp/chatapp4.png b/docs/create/aws/chatapp/chatapp4.png new file mode 100644 index 0000000000..cf3dbb13de Binary files /dev/null and b/docs/create/aws/chatapp/chatapp4.png differ diff --git a/docs/create/aws/chatapp/chatapp_homepage.png b/docs/create/aws/chatapp/chatapp_homepage.png new file mode 100644 index 0000000000..3967d54859 Binary files /dev/null and b/docs/create/aws/chatapp/chatapp_homepage.png differ diff --git a/docs/create/aws/chatapp/chatapp_server.png b/docs/create/aws/chatapp/chatapp_server.png new file mode 100644 index 0000000000..b5c141478e Binary files /dev/null and b/docs/create/aws/chatapp/chatapp_server.png differ diff --git a/docs/create/aws/chatapp/image_chatapp1.png b/docs/create/aws/chatapp/image_chatapp1.png new file mode 100644 index 0000000000..a995f8237d Binary files /dev/null and b/docs/create/aws/chatapp/image_chatapp1.png differ diff --git a/docs/create/aws/chatapp/index-chatapp.mdx b/docs/create/aws/chatapp/index-chatapp.mdx new file mode 100644 index 0000000000..0724fba26d --- /dev/null +++ b/docs/create/aws/chatapp/index-chatapp.mdx @@ -0,0 +1,267 @@ +--- +id: index-chatapp +title: How to Build a Real-Time Chat application on AWS using Python and Redis +sidebar_label: Building a Real-Time Chat application on AWS using Python and Redis +slug: /create/aws/chatapp +authors: [ajeet] +--- + +Chat messaging apps are surging in popularity exponentially. Mobile apps like WhatsApp, Facebook, Telegram, Slack, Discord have become “a part and parcel” of our life. Users are addicted to these apps as they bring a personal touch and offer a real-time interaction. + +![chatapp](image_chatapp1.png) + +There’s been a rise in the number of social media apps that bring social elements to enable activities like collaboration, messaging, social interaction and commenting. Such activities require a real-time capability to automatically present updated information to users. More and more developers are tapping into the power of Redis as it is extremely fast & its support for a variety of rich data structures such as Lists, Sets, Sorted Sets, Hashes etc. Redis comes along with a Pub/Sub messaging functionality that allows developers to scale the backend by spawning multiple server instances. + + + +- What will you build? +- What will you need? +- Getting started +- How it works? +- How is the data stored? +- How is the data accessed? + +### 1. What will you build? + +In this tutorial, we will see how to build a basic chat application built with Flask, Socket.IO and Redis Enterprise Cloud running on AWS Cloud. This example uses pub/sub features combined with web-sockets for implementing the message communication between client and server. + +![image](chatapp2.png) + +### 2. What will you need? + +- Frontend - React, Socket.IO +- Backend - Python(Flask), Redis Enterprise Cloud hosted on AWS + +### 3. Getting Started + + +### Step 1. Sign up for a Free Redis Enterprise Cloud Account + +[Follow this tutorial](https://developer.redis.com/create/aws/redis-on-aws) to sign up for a free Redis Enterprise Cloud account. + +![image](chatapp3.png) + +Choose AWS as a Cloud vendor while creating your new subscription. At the end of the database creation process, you will get a Redis Enterprise CLoud database endpoint and password. You can save it for later use. + +![image](chatapp4.png) + +### Step 2. Clone the repository + +``` +git clone https://github.com/redis-developer/basic-redis-chat-app-demo-python +``` + + +### Step 3. Installing the requred packages + +``` +cd client +yarn install +``` + +### Step 4. Starting the frontend + +``` +yarn start +``` + +``` +You can now view client in the browser. + + Local: http://localhost:3000 + On Your Network: http://192.168.1.9:3000 +``` + + +![chatapp](chatapp_server.png) + + + +### Step 5. Installing the required Python modules + + +``` +cd .. +pip3 install -r requirements.txt +``` + +### Step 6: Running Backend + + + + + +``` +python3 -m venv venv/ +source venv/bin/activate +python3 app.py +``` + +``` +python3 app.py + * Restarting with stat + * Debugger is active! + * Debugger PIN: 220-696-610 +(8122) wsgi starting up on http://127.0.0.1:5000 +``` + +![Chatapp](chatapp_homepage.png) + +### Step 7. How it works? + + +The chat server works as a basic REST API which involves keeping the session and handling the user state in the chat rooms (besides the WebSocket/real-time part). +When the server starts, the initialization step occurs. At first, a new Redis connection is established and it's checked whether it's needed to load the demo data. + +#### Initialization + +For simplicity, a key with total_users value is checked: if it does not exist, we fill the Redis database with initial data. EXISTS total_users (checks if the key exists) +The demo data initialization is handled in multiple steps: + +#### Creating demo users + +We create a new user id: INCR total_users. Then we set a user ID lookup key by user name: e.g. + +``` +SET username:nick user:1 +``` + +And finally, the rest of the data is written to the hash set: + +Example: + + ```bash + HSET user:1 username "nick" password "bcrypt_hashed_password". + ``` + +Additionally, each user is added to the default "General" room. +For handling rooms for each user, we have a set that holds the room ids. Here's an example command of how to add the room: + + ```bash + SADD user:1:rooms "0" + ``` + +#### Populating private messages between users + +First, private rooms are created: if a private room needs to be established, for each user a room id: room:1:2 is generated, where numbers correspond to the user ids in ascending order. + +E.g. Create a private room between 2 users: + + ```bash + SADD user:1:rooms 1:2 and SADD user:2:rooms 1:2 + ``` + +Then we add messages to this room by writing to a sorted set: + + ```bash + ZADD room:1:2 1615480369 "{'from': 1, 'date': 1615480369, 'message': 'Hello', 'roomId': '1:2'}" + ``` + + +We are using a stringified JSON to keep the message structure and simplify the implementation details for this demo-app. You may choose to use a Hash or RedisJSON + +### Populate the "General" room with messages + +Messages are added to the sorted set with id of the "General" room: room:0 + +#### Pub/sub + +After initialization, a pub/sub subscription is created: SUBSCRIBE MESSAGES. At the same time, each server instance will run a listener on a message on this channel to receive real-time updates. + +Again, for simplicity, each message is serialized to JSON, which we parse and then handle in the same manner, as WebSocket messages. + +Pub/sub allows connecting multiple servers written in different platforms without taking into consideration the implementation detail of each server. + +#### Real-time chat and session handling + +When a WebSocket connection is established, we can start to listen for events: +- Connection. A new user is connected. At this point, a user ID is captured and saved to the session (which is cached in Redis). Note, that session caching is language/library-specific and it's used here purely for persistence and maintaining the state between server reloads. + +A global set with online_users key is used for keeping the online state for each user. So on a new connection, a user ID is written to that set: + + ```bash + SADD online_users 1 + ``` + +Here we have added user with id 1 to the set online_users + +After that, a message is broadcasted to the clients to notify them that a new user is joined the chat. + +- Disconnect. It works similarly to the connection event, except we need to remove the user for online_users set and notify the clients: SREM online_users 1 (makes user with id 1 offline). + +- Message. A user sends a message, and it needs to be broadcasted to the other clients. The pub/sub allows us also to broadcast this message to all server instances which are connected to this Redis: + + ``` + PUBLISH message "{'serverId': 4132, 'type':'message', 'data': {'from': 1, 'date': 1615480369, 'message': 'Hello', 'roomId': '1:2'}}" + ``` + +Note we send additional data related to the type of the message and the server id. Server id is used to discard the messages by the server instance which sends them since it is connected to the same MESSAGES channel. + +The type field of the serialized JSON corresponds to the real-time method we use for real-time communication (connect/disconnect/message). + +The data is method-specific information. In the example above it's related to the new message. + +### Step 8. How the data is stored? + +Redis is used mainly as a database to keep the user/messages data and for sending messages between connected servers. + +The real-time functionality is handled by Socket.IO for server-client messaging. Additionally each server instance subscribes to the MESSAGES channel of pub/sub and dispatches messages once they arrive. Note that, the server transports pub/sub messages with a separate event stream (handled by Server Sent Events), this is due to the need of running pub/sub message loop apart from socket.io signals. + +The chat data is stored in various keys and various data types. +User data is stored in a hash set where each user entry contains the next values: +- username: unique user name; +- password: hashed password + +- Additionally a set of rooms is associated with user +- Rooms are sorted sets which contains messages where score is the timestamp for each message +- Each room has a name associated with it +- Online set is global for all users is used for keeping track on which user is online. +- User hash set is accessed by key user:{userId}. The data for it stored with HSET key field data. User id is calculated by incrementing the total_users key (INCR total_users) + +- Username is stored as a separate key (username:{username}) which returns the userId for quicker access and stored with SET username:{username} {userId}. + +- Rooms which user belongs too are stored at user:{userId}:rooms as a set of room ids. A room is added by SADD user:{userId}:rooms {roomId} command. + +- Messages are stored at room:{roomId} key in a sorted set (as mentioned above). They are added with ZADD room:{roomId} {timestamp} {message} command. Message is serialized to an app-specific JSON string. + +### Step 9. How the data is accessed? + +Get User HGETALL user:{id}. + + ```bash + HGETALL user:2 + ``` + +where we get data for the user with id: 2. + +- Online users: SMEMBERS online_users. This will return ids of users which are online + +- Get room ids of a user: SMEMBERS user:{id}:rooms. +Example: + + ``` + SMEMBERS user:2:rooms + ``` + +This will return IDs of rooms for user with ID: 2 + +- Get list of messages ZREVRANGE room:{roomId} {offset_start} {offset_end}. +Example: + + ``` + ZREVRANGE room:1:2 0 50 + ``` +It will return 50 messages with 0 offsets for the private room between users with IDs 1 and 2. + + + +### Further References + +- [Building a sample Redis Chat application demo in .Net](https://github.com/redis-developer/basic-redis-chat-app-demo-dotnet) +- [Building a Sample Redis Chat application demo in Java](https://github.com/redis-developer/basic-redis-chat-app-demo-java) +- [Building a Sample Redis Chat application demo in NodeJS](https://github.com/redis-developer/basic-redis-chat-app-demo-nodejs) +- [Building a Sample Redis Chat application demo in Go](https://github.com/redis-developer/basic-redis-chat-demo-go) +- [Building a Sample Redis Chat application demo in Ruby](https://github.com/redis-developer/basic-redis-chat-demo-ruby) + + + diff --git a/docs/create/aws/chatapp/index-chatapp.mdx_python b/docs/create/aws/chatapp/index-chatapp.mdx_python new file mode 100644 index 0000000000..d50e68bc2b --- /dev/null +++ b/docs/create/aws/chatapp/index-chatapp.mdx_python @@ -0,0 +1,247 @@ +--- +id: index-chatapp +title: How to build a Chat application using Redis +sidebar_label: How to build a Chat application using Redis +slug: /howtos/chatapp +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import useBaseUrl from '@docusaurus/useBaseUrl'; +import RedisCard from '@site/src/theme/RedisCard'; + + +In this tutorial, we will see how to build a basic chat application built with Flask, Socket.IO and Redis. + +![Chatapp](chatapp_homepage.png) + + + + +### Step 1. Pre-requisite + +- Python 3.6+ + +### Step 2. Clone the repository + +``` +git clone https://github.com/redis-developer/basic-redis-chat-app-demo-python +``` + + +### Step 3. Installing the requred packages + +``` +cd client +yarn install +``` + +### Step 4. Starting the frontend + +``` +yarn start +``` + +``` +You can now view client in the browser. + + Local: http://localhost:3000 + On Your Network: http://192.168.1.9:3000 +``` + + +![chatapp](chatapp_server.png) + + + +### Step 4. Installing the required Python modules + + +``` +cd .. +pip3 install -r requirements.txt +``` + +### Step 5: Running Backend + + + + + +``` +python3 -m venv venv/ +source venv/bin/activate +python3 app.py +``` + +``` +python3 app.py + * Restarting with stat + * Debugger is active! + * Debugger PIN: 220-696-610 +(8122) wsgi starting up on http://127.0.0.1:5000 +``` + +![Chatapp](chatapp_homepage.png) + +### How it works? + + +The chat server works as a basic REST API which involves keeping the session and handling the user state in the chat rooms (besides the WebSocket/real-time part). +When the server starts, the initialization step occurs. At first, a new Redis connection is established and it's checked whether it's needed to load the demo data. + +#### Initialization + +For simplicity, a key with total_users value is checked: if it does not exist, we fill the Redis database with initial data. EXISTS total_users (checks if the key exists) +The demo data initialization is handled in multiple steps: + +#### Creating of demo users + +We create a new user id: INCR total_users. Then we set a user ID lookup key by user name: e.g. + +``` +SET username:nick user:1 +``` + +And finally, the rest of the data is written to the hash set: e.g. HSET user:1 username "nick" password "bcrypt_hashed_password". + +Additionally, each user is added to the default "General" room. +For handling rooms for each user, we have a set that holds the room ids. Here's an example command of how to add the room: + +``` +SADD user:1:rooms "0" +``` + +Populate private messages between users. At first, private rooms are created: if a private room needs to be established, for each user a room id: room:1:2 is generated, where numbers correspond to the user ids in ascending order. + +E.g. Create a private room between 2 users: + +``` +SADD user:1:rooms 1:2 and SADD user:2:rooms 1:2 +``` + +Then we add messages to this room by writing to a sorted set: + +``` +ZADD room:1:2 1615480369 "{'from': 1, 'date': 1615480369, 'message': 'Hello', 'roomId': '1:2'}" +``` +We use a stringified JSON for keeping the message structure and simplify the implementation details for this demo-app. + +Populate the "General" room with messages. Messages are added to the sorted set with id of the "General" room: room:0 + +#### Pub/sub + +After initialization, a pub/sub subscription is created: SUBSCRIBE MESSAGES. At the same time, each server instance will run a listener on a message on this channel to receive real-time updates. + +Again, for simplicity, each message is serialized to JSON, which we parse and then handle in the same manner, as WebSocket messages. + +Pub/sub allows connecting multiple servers written in different platforms without taking into consideration the implementation detail of each server. + +#### Real-time chat and session handling + +When a WebSocket/real-time server is instantiated, which listens for the next events: + +- Connection. A new user is connected. At this point, a user ID is captured and saved to the session (which is cached in Redis). Note, that session caching is language/library-specific and it's used here purely for persistence and maintaining the state between server reloads. + +A global set with online_users key is used for keeping the online state for each user. So on a new connection, a user ID is written to that set: + +``` +SADD online_users 1 +``` + +Here we have added user with id 1 to the set online_users + +After that, a message is broadcasted to the clients to notify them that a new user is joined the chat. + +- Disconnect. It works similarly to the connection event, except we need to remove the user for online_users set and notify the clients: SREM online_users 1 (makes user with id 1 offline). + +- Message. A user sends a message, and it needs to be broadcasted to the other clients. The pub/sub allows us also to broadcast this message to all server instances which are connected to this Redis: + +``` +PUBLISH message "{'serverId': 4132, 'type':'message', 'data': {'from': 1, 'date': 1615480369, 'message': 'Hello', 'roomId': '1:2'}}" +``` + +Note we send additional data related to the type of the message and the server id. Server id is used to discard the messages by the server instance which sends them since it is connected to the same MESSAGES channel. + +type field of the serialized JSON corresponds to the real-time method we use for real-time communication (connect/disconnect/message). + +data is method-specific information. In the example above it's related to the new message. + +### How the data is stored: + +Redis is used mainly as a database to keep the user/messages data and for sending messages between connected servers. + +The real-time functionality is handled by Socket.IO for server-client messaging. Additionally each server instance subscribes to the MESSAGES channel of pub/sub and dispatches messages once they arrive. Note that, the server transports pub/sub messages with a separate event stream (handled by Server Sent Events), this is due to the need of running pub/sub message loop apart from socket.io signals. + +The chat data is stored in various keys and various data types. +User data is stored in a hash set where each user entry contains the next values: +- username: unique user name; +- password: hashed password + +- Additionally a set of rooms is associated with user +- Rooms are sorted sets which contains messages where score is the timestamp for each message +- Each room has a name associated with it +- Online set is global for all users is used for keeping track on which user is online. +- User hash set is accessed by key user:{userId}. The data for it stored with HSET key field data. User id is calculated by incrementing the total_users key (INCR total_users) + +- Username is stored as a separate key (username:{username}) which returns the userId for quicker access and stored with SET username:{username} {userId}. + +- Rooms which user belongs too are stored at user:{userId}:rooms as a set of room ids. A room is added by SADD user:{userId}:rooms {roomId} command. + +- Messages are stored at room:{roomId} key in a sorted set (as mentioned above). They are added with ZADD room:{roomId} {timestamp} {message} command. Message is serialized to an app-specific JSON string. + +### How the data is accessed: + +Get User HGETALL user:{id}. + +``` +HGETALL user:2 +``` + +where we get data for the user with id: 2. + +- Online users: SMEMBERS online_users. This will return ids of users which are online + +- Get room ids of a user: SMEMBERS user:{id}:rooms. +Example: + +``` +SMEMBERS user:2:rooms +``` + +This will return IDs of rooms for user with ID: 2 + +- Get list of messages ZREVRANGE room:{roomId} {offset_start} {offset_end}. +Example: + +``` +ZREVRANGE room:1:2 0 50 +``` +It will return 50 messages with 0 offsets for the private room between users with IDs 1 and 2. + + + + + + + + + + + + + +## Step 1. Pre-requisite + + + + + + + diff --git a/docs/create/aws/index-aws.mdx b/docs/create/aws/index-aws.mdx index d53114817a..963b094b0d 100644 --- a/docs/create/aws/index-aws.mdx +++ b/docs/create/aws/index-aws.mdx @@ -12,7 +12,7 @@ The following links provide you with the available options to run apps on Heroku
@@ -27,10 +27,34 @@ The following links provide you with the available options to run apps on Heroku
+
+ +
+
+ +
+
+ +
+
+ +
diff --git a/docs/create/openshift/create_rec.png b/docs/create/openshift/create_rec.png new file mode 100644 index 0000000000..8f59d9bc41 Binary files /dev/null and b/docs/create/openshift/create_rec.png differ diff --git a/docs/create/openshift/image_03.png b/docs/create/openshift/image_03.png new file mode 100644 index 0000000000..22af2a1546 Binary files /dev/null and b/docs/create/openshift/image_03.png differ diff --git a/docs/create/openshift/image_04.png b/docs/create/openshift/image_04.png new file mode 100644 index 0000000000..14f8de773f Binary files /dev/null and b/docs/create/openshift/image_04.png differ diff --git a/docs/create/openshift/image_05.png b/docs/create/openshift/image_05.png new file mode 100644 index 0000000000..2a296df0a5 Binary files /dev/null and b/docs/create/openshift/image_05.png differ diff --git a/docs/create/openshift/image_1.png b/docs/create/openshift/image_1.png new file mode 100644 index 0000000000..c8a9df2f78 Binary files /dev/null and b/docs/create/openshift/image_1.png differ diff --git a/docs/create/openshift/image_login.png b/docs/create/openshift/image_login.png new file mode 100644 index 0000000000..50d2801740 Binary files /dev/null and b/docs/create/openshift/image_login.png differ diff --git a/docs/create/openshift/index-openshift.mdx b/docs/create/openshift/index-openshift.mdx new file mode 100644 index 0000000000..0f6d14137e --- /dev/null +++ b/docs/create/openshift/index-openshift.mdx @@ -0,0 +1,428 @@ +--- +id: index-openshift +title: How to deploy Redis Enterprise on Red Hat OpenShift Container Platform via OperatorHub +sidebar_label: Redis on Red Hat OpenShift Container Platform +slug: /create/openshift +authors: [karan,sumit,ajeet] +--- + +Deploying and managing containerised applications is not easy. With the rise of microservice architecture, it becomes cumbersome to deploy containers across multiple environments. Given that containers can be spun up in seconds and that too in a higher volume compared to VMs, managing across multiple platforms can be extremely challenging. OpenShift (kubernetes) comes to the rescue. Kubernetes is designed and built from the ground up to deploy and manage containerized applications across hundreds of compute nodes. Red Hat OpenShift is a leading enterprise kubernetes platform that enables a cloud-like experience everywhere it’s deployed, no matter if it’s in the cloud, on-premise or at the edge. + +### What is OpenShift? + +[Red Hat OpenShift](https://www.redhat.com/en/technologies/cloud-computing/openshift) gives you the ability to choose where you build, deploy, and run applications through a consistent experience. Red Hat OpenShift’s full-stack automated operations and self-service provisioning for developers lets teams work together more efficiently to move ideas from development to production. + +[Red Hat OpenShift focused on developer experience](https://www.redhat.com/en/technologies/cloud-computing/openshift/developers) and application security that's platform agnostic. OpenShift helps you develop and deploy applications to one or more hosts. These can be public facing web applications, or backend applications, including micro services or databases. Applications can be implemented in any programming language you choose. The only requirement is that the application can run within a container. + +[OpenShift Container Platform(OCP)](https://www.redhat.com/en/technologies/cloud-computing/openshift/container-platform) is a set of modular components and services built on top of Red Hat Enterprise Linux, and Kubernetes. OpenShift adds capabilities such as remote management, multitenancy, increased security, application life-cycle management and self-service interfaces for developers. + +Red Hat OpenShift is built on Kubernetes and extends the open source platform in several ways to make it suitable for enterprise operations teams: +- Automated installation and operations for platform updates on the container host, the Kubernetes cluster, and application services +- A consistent Kubernetes platform portable across different infrastructures and clouds +- Built-in cluster and application monitoring using Prometheus and Grafana dashboards +- Centralized policy management and enforcement across clusters +- Built-in security checks for the entire container stack throughout the application life cycle + + +### Supported Technologies + +OpenShift is fully pluggable and works with a wide range of technologies some of them are Languages Runtimes like .NET, Java, Node.js, PHP, Python, Ruby, GoLang and Perl. Databases such as MariaDB, MongoDB, MySQL, PostgreSQL, and Redis. Red Hat and IBM Application Services which includes application runtimes, agile integration, business process automation technologies and other enterprise software solutions are available as containers which are managed by Operators in OpenShift. Also, OpenShift is certified with and supports 100+ independent software vendors including Redis Enterprise. + +### What is OperatorHub? + +OperatorHub is the web console interface in OpenShift Container Platform that cluster administrators use to discover, install and manage Operators. In a few clicks, users can subscribe to the software they like via operators and make it available to software developers and platform administrators across cluster or isolated namespace environments using Operator Lifecycle Manager (OLM). + +In this tutorial, you will learn how to deploy Redis Enterprise Cluster and Database on OpenShift using Redis Enterprise Kubernetes Operator. + +### Pre-requisites: + +To follow this guide, you should have an OpenShift cluster with 3 or more worker nodes. There are multiple ways to provision an OpenShift cluster, [you can follow the installation guide](https://docs.openshift.com/container-platform/4.9/installing/index.html) based on your preferred infrastructure or cloud provider. Redis Enterprise Operator currently supports OpenShift 3.11 and OpenShift 4.x. [Visit this page](https://docs.redis.com/latest/kubernetes/reference/supported_k8s_distributions/) to find all the supported Kubernetes distributions. + +In this guide we are using a self-managed OpenShift Cluster on Google Cloud Platform. + + +### Step 1. Login to Red Hat Openshift Container Platform(OCP) + +You can login to OCP via CLI and user with cluster-admin role. + + +``` +$ oc login +Authentication required for https://api.sumit.openshift.demo.redislabs.com:6443 (openshift) +Username: +Password: +Login successful. +``` + +You have access to 68 projects, the list has been suppressed. You can list all projects with 'oc projects'. + +### Step 2. Verify the status of nodes + +All the nodes should be in “Ready” status. + + +``` +$ oc get nodes +NAME STATUS ROLES AGE VERSION +demoopenshift-9x5xq-infra-b-h28tc Ready infra,worker 86m v1.22.3+e790d7f +demoopenshift-9x5xq-infra-b-wrfgg Ready infra,worker 85m v1.22.3+e790d7f +demoopenshift-9x5xq-master-0 Ready master 103m v1.22.3+e790d7f +demoopenshift-9x5xq-master-1 Ready master 103m v1.22.3+e790d7f +demoopenshift-9x5xq-master-2 Ready master 103m v1.22.3+e790d7f +demoopenshift-9x5xq-worker-b-6jwx2 Ready worker 97m v1.22.3+e790d7f +demoopenshift-9x5xq-worker-b-9pxhd Ready worker 95m v1.22.3+e790d7f +``` + + + +### Step 3. Create a New Project + +Redis Enterprise can only be deployed in an user’s custom project. Hence, let us create a new project. + + +``` +$ oc new-project myproject +Now using project "myproject" on server "https://api.sumit.openshift.demo.redislabs.com:6443". + +You can add applications to this project with the 'new-app' command. For example, try: + + oc new-app ruby~https://github.com/sclorg/ruby-ex.git + +to build a new example application in Ruby. Or use kubectl to deploy a simple Kubernetes application: + + kubectl create deployment hello-node --image=gcr.io/hello-minikube-zero-install/hello-node +``` + + + +### Step 4. Clone the Redis Enterprise on Kubernetes Github repository + + +``` +$ git clone https://github.com/RedisLabs/redis-enterprise-k8s-docs +$ cd redis-enteprise-k8s-docs/ +``` + +### Step 5. Apply the SCC configuration + + +The scc [(Security Context Constraint)](https://docs.openshift.com/container-platform/4.8/authentication/managing-security-context-constraints.html) yaml defines security context constraints for the cluster for our project. This is composed of custom security settings and strategies that will provide access to the Redis Enterprise Operator and Cluster Pods to run on OpenShift. We strongly recommend that you not change anything in this yaml file. + +``` +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: redis-enterprise-scc +allowPrivilegedContainer: false +allowedCapabilities: + - SYS_RESOURCE +runAsUser: + type: MustRunAs + uid: 1001 +FSGroup: + type: MustRunAs + ranges: 1001,1001 +seLinuxContext: + type: RunAsAny +``` + +Apply the configuration to a resource by filename “scc.yaml” which is available in “openshift” folder as shown below: + + +``` + $ oc apply -f scc.yaml + securitycontextconstraints.security.openshift.io/redis-enterprise-scc configured +``` + +### Step 6. Provide the operator permissions for Redis Enterprise Operator and Cluster pods + +``` + $ oc adm policy add-scc-to-user redis-enterprise-scc system:serviceaccount:myproject:redis-enterprise-operator + $ oc adm policy add-scc-to-user redis-enterprise-scc system:serviceaccount:myproject:rec +``` + + +### Step 7. Deploy Redis Enterprise Operator + +If you want to perform this operation via UI, follow the below steps: + +Login to OpenShift console with admin privileges. + +![image](image_login.png) + +Click on “OperatorHub” under Operators section. + +![image](image_03.png) + + + +This will open up a list of operators with various categories. Search for “Redis” and click “Redis Enterprise” without “Marketplace” tag to proceed further. + + + +![image](image_04.png) + + + + +### Step 8. Install Redis Enterprise Operator + +Click “Install” to start the installation process and then select the required channel, namespace and approval strategy in the next screen. + + +![image](image_05.png) + + +Wait till the installation gets completed. + +![image](installed_operators.png) + + +Click on “Events” to see the streaming events to troubleshoot or check the progress of installation. + +![image](streaming_events.png) + + + +Once Redis Enterprise Operator is installed, we will create a Redis Enterprise Cluster instance using RedisEnterpriseCluster (REC) custom resource definition. + +RedisEnterpriseCluster is the Schema for the redisenterpriseclusters API. From the installed Redis EnterpriseOperator, Click on “Create Redis Enterprise Cluster'' and click on “YAML” section and paste below sample yaml file: + + +![image](create_rec.png) + + +``` +apiVersion: app.redislabs.com/v1 +kind: RedisEnterpriseCluster +metadata: + name: rec + namespace: myproject +spec: + redisEnterpriseNodeResources: + limits: + cpu: 4000m + memory: 4Gi + requests: + cpu: 4000m + memory: 4Gi + persistentSpec: + enabled: true + storageClassName: standard + redisEnterpriseImageSpec: + imagePullPolicy: IfNotPresent + repository: redislabs/redis + versionTag: 6.2.8-64.rhel7-openshift + servicesRiggerSpec: + databaseServiceType: load_balancer + serviceNaming: redis-port + nodes: 3 + uiServiceType: LoadBalancer + username: demo@redis.com +``` + +For configuring any additional parameters for the Redis Enterprise Cluster custom resource definition, visit this page. + +Wait till the status shows “Succeeded” under Redis Enterprise Operator. + +![image](succeeded.png) + +### Step 9. Prerequisites to access Redis Enterprise Cluster Admin Console + +To access the Redis Enterprise Cluster admin console, you will need credentials. Follow the below steps to get the credentials via “rec” secret resource: + +![image](ocp.png) + +Under the “Data” section, you can reveal username and password. + +Next, Click on “Services” and choose “rec-ui” under the services list: + +![image](recui.png) + +Click on “Services” and look out for External Load-balancer IP. + +![image](loadbalancer.png) + + +![image](recui2.png) + +### Step 10. Access the Redis Enterprise Cluster Amdin Console + +From web browser access the rec-ui external IP http://IP:8443 to access the Redis Enterprise Cluster Admin Console. + +![image](re1.png) + +To login into Redis Enterprise Cluster, use the credentials what we have extracted above. +By now, you will be able to see 3 nodes of the Redis Enterprise cluster. + +![image](re2.png) + + + + +### Step 11. Create Redis Enterprise Database + +RedisEnterpriseDatabase is the Schema for the redisenterprisedatabases API. To create REDB, navigate to Installed Operator > Redis Enterprise Operator > Redis Enterprise Database > Click “Create RedisEnterpriseDatabase” to start the creation process using REDB custom resource. You can click on “YAML” and paste below sample yaml file. + +``` +apiVersion: app.redislabs.com/v1alpha1 +kind: RedisEnterpriseDatabase +metadata: + name: redb + namespace: myproject +spec: + redisEnterpriseCluster: + name: rec + memorySize: 1GB + tlsMode: enabled +``` + + +For configuring any additional parameters for the Redis Enterprise Database custom resource, visit this page. +Once the REDB creation is done, you should see the status as “active”. + +![Image](re3.png) + +Finally, navigate to Redis Enterprise Cluster admin console to verify Redis Enterprise database. + +![Image](re4.png) + +Click on the newly created database and then click on configuration option to see the overall details: + +![Image](re6.png) + + +Navigate back to the OpenShift console and verify the service object created for the Redis Database. + +![Image](re7.png) + + + +Select “Routes” from left panel and create a new route object for this database service with below sample yaml file: + + +``` +kind: Route +apiVersion: route.openshift.io/v1 +metadata: + name: redb + labels: + app: redis-enterprise + redis.io/cluster: rec +spec: + host: redb-re-test.apps.sumit.openshift.demo.redislabs.com + to: + kind: Service + name: redis-12119-load-balancer + weight: 100 + port: + targetPort: redis + tls: + termination: passthrough + insecureEdgeTerminationPolicy: None +``` + + + + +Note the Host name 'my-db-hostname' from the created route object. + +![image](re8.png) + + + +Note the database password "my-db-password" from the database secret object with name “db-name-redb”. + +![image](re9.png) + + +Get the default CA certificate from the redis-enterprise-node container on any of the Redis Enterprise pods and paste the output in a file called “proxy_cert.pem”. + +``` +$ oc exec -it -c redis-enterprise-node -- cat \ /etc/opt/redislabs/proxy_cert.pem +``` + + +Run the following openssl command, substituting your own values for 'my-db-hostname'. + +``` +$ openssl s_client -connect :443 -crlf \ + -CAfile ./proxy_cert.pem -servername +``` + + + +Authorize the database with the password and enter the ping command. If you are connected to the database, you will receive PONG back, as shown below: + +``` +... +... +... + Verify return code: 0 (ok) + --- + auth 'my-db-password' + +OK + + PING + +PONG +``` + + + + +You can use the code below to test your access with Python, substituting your own values for 'my-db-hostname' and 'file-path'. + +``` + import redis + try: + r = redis.StrictRedis( + host='my-db-hostname', + port=443, + password='my-db-password', + ssl=True, + ssl_ca_certs='file-path') + print (r.info()) + #test the connection + print('Testing redis by setting a key and value..') + r.set('hello', 'world') + value = r.get('hello') + print(value) + except Exception as e: + print (e) +``` + + + + +Your output should look something like this: + +``` +{'redis_version': '6.0.13', 'redis_git_sha1': 0, 'redis_git_dirty': 0, 'redis_build_id': 0, 'redis_mode': 'standalone', 'os': 'Linux 4.18.0-305.30.1.el8_4.x86_64 x86_64' +... +... +... + +Testing redis by setting a key and value.. +b'world' + +``` + + + + + +In our next tutorial, we will learn how to deploy a sample Real-time chat application powered by Go and Redis on OpenShift Container Platform. + + +### References: + +- [Deploy Redis Enterprise Software on Kubernetes with OpenShift](https://docs.redis.com/latest/kubernetes/deployment/openshift/) +- [Official OpenShift Documentation](https://docs.openshift.com/) +- [Install a Cluster quickly on GCP](https://docs.openshift.com/container-platform/4.7/installing/installing_gcp/installing-gcp-default.html) +- [Redis Enterprise on Kubernetes Github Repository](https://github.com/RedisLabs/redis-enterprise-k8s-docs) + + + + + + diff --git a/docs/create/openshift/installed_operators.png b/docs/create/openshift/installed_operators.png new file mode 100644 index 0000000000..3042a50e61 Binary files /dev/null and b/docs/create/openshift/installed_operators.png differ diff --git a/docs/create/openshift/loadbalancer.png b/docs/create/openshift/loadbalancer.png new file mode 100644 index 0000000000..3ec213756d Binary files /dev/null and b/docs/create/openshift/loadbalancer.png differ diff --git a/docs/create/openshift/ocp.png b/docs/create/openshift/ocp.png new file mode 100644 index 0000000000..f59b82ed1e Binary files /dev/null and b/docs/create/openshift/ocp.png differ diff --git a/docs/create/openshift/re1.png b/docs/create/openshift/re1.png new file mode 100644 index 0000000000..decd89c886 Binary files /dev/null and b/docs/create/openshift/re1.png differ diff --git a/docs/create/openshift/re2.png b/docs/create/openshift/re2.png new file mode 100644 index 0000000000..1f31c0006c Binary files /dev/null and b/docs/create/openshift/re2.png differ diff --git a/docs/create/openshift/re3.png b/docs/create/openshift/re3.png new file mode 100644 index 0000000000..991139a51f Binary files /dev/null and b/docs/create/openshift/re3.png differ diff --git a/docs/create/openshift/re4.png b/docs/create/openshift/re4.png new file mode 100644 index 0000000000..1ec38a0878 Binary files /dev/null and b/docs/create/openshift/re4.png differ diff --git a/docs/create/openshift/re5.png b/docs/create/openshift/re5.png new file mode 100644 index 0000000000..cda9efefd4 Binary files /dev/null and b/docs/create/openshift/re5.png differ diff --git a/docs/create/openshift/re6.png b/docs/create/openshift/re6.png new file mode 100644 index 0000000000..45c7b8d69b Binary files /dev/null and b/docs/create/openshift/re6.png differ diff --git a/docs/create/openshift/re7.png b/docs/create/openshift/re7.png new file mode 100644 index 0000000000..c4fbd70461 Binary files /dev/null and b/docs/create/openshift/re7.png differ diff --git a/docs/create/openshift/re8.png b/docs/create/openshift/re8.png new file mode 100644 index 0000000000..e52f0c7caa Binary files /dev/null and b/docs/create/openshift/re8.png differ diff --git a/docs/create/openshift/re9.png b/docs/create/openshift/re9.png new file mode 100644 index 0000000000..b0b4113b5a Binary files /dev/null and b/docs/create/openshift/re9.png differ diff --git a/docs/create/openshift/recui.png b/docs/create/openshift/recui.png new file mode 100644 index 0000000000..847674ae7b Binary files /dev/null and b/docs/create/openshift/recui.png differ diff --git a/docs/create/openshift/recui2.png b/docs/create/openshift/recui2.png new file mode 100644 index 0000000000..201d089669 Binary files /dev/null and b/docs/create/openshift/recui2.png differ diff --git a/docs/create/openshift/streaming_events.png b/docs/create/openshift/streaming_events.png new file mode 100644 index 0000000000..3b0c6bf9ae Binary files /dev/null and b/docs/create/openshift/streaming_events.png differ diff --git a/docs/create/openshift/succeeded.png b/docs/create/openshift/succeeded.png new file mode 100644 index 0000000000..855c0a93f2 Binary files /dev/null and b/docs/create/openshift/succeeded.png differ diff --git a/docusaurus.config.js b/docusaurus.config.js index 723c9f63cd..0993339df1 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -28,7 +28,7 @@ module.exports = { ajeet: { name: 'Ajeet Raina', link: 'https://twitter.com/ajeetsraina', - title: 'Developer Marketing Manager at Redis', + title: 'Developer Growth Manager at Redis', image: 'profile_pic_ajeet_raina.jpg' }, will: { @@ -73,6 +73,18 @@ module.exports = { title: 'Principal Field Engineer at Redis', image: 'profile_pic_christian_mague.jpg', }, + karan: { + name: 'Karan Singh', + link: 'https://www.linkedin.com/in/karansingh7', + title: 'Senior Principal Architect & Developer Evangelist at Red Hat', + image: 'profile_pic_karan_singh.png', + }, + sumit: { + name: 'Sumit Shatwara', + link: 'https://www.linkedin.com/in/sumitshatwara', + title: 'Solution Architect at Redis', + image: 'profile_pic_sumit_shatwara.jpeg', + }, matthew: { name: 'Matthew Royal', link: 'https://www.linkedin.com/in/royalmatthew/', diff --git a/sidebars.js b/sidebars.js index b7697e2043..3ffe4683f6 100644 --- a/sidebars.js +++ b/sidebars.js @@ -44,8 +44,10 @@ module.exports = { 'create/aws/redis-on-aws/index-redis-on-aws', 'create/aws/slackbot/index-slackbot', 'create/aws/terraform/index-terraform', - 'create/aws/import/index-database-migration-aws-elasticache-redis-enterprise-cloud' - ] + 'create/aws/bidding-on-aws/index-bidding-on-aws', + 'create/aws/import/index-database-migration-aws-elasticache-redis-enterprise-cloud', + 'create/aws/chatapp/index-chatapp' + ] }, 'create/docker/index-docker', { @@ -61,7 +63,8 @@ module.exports = { 'create/windows/index-windows', 'create/from-source/index-from-source', 'create/jenkins/index-jenkins', - ] + 'create/openshift/index-openshift' + ] }, { type: 'category', diff --git a/static/img/matrix200.png b/static/img/matrix200.png new file mode 100644 index 0000000000..0465899283 Binary files /dev/null and b/static/img/matrix200.png differ diff --git a/static/img/profile_pic_karan_singh.png b/static/img/profile_pic_karan_singh.png new file mode 100644 index 0000000000..1e196ddcb6 Binary files /dev/null and b/static/img/profile_pic_karan_singh.png differ diff --git a/static/img/profile_pic_sumit_shatwara.jpeg b/static/img/profile_pic_sumit_shatwara.jpeg new file mode 100644 index 0000000000..56a65a9bf3 Binary files /dev/null and b/static/img/profile_pic_sumit_shatwara.jpeg differ