first commit

.dockerignore

@@ -0,0 +1,3 @@
+Dockerfile
+.git
+.env

.env.example

@@ -0,0 +1,10 @@
+MONGODB_URL=
+DATABASE=
+
+CONTENT_COUNT=
+
+GITHUB_TOKEN=
+GITHUB_USERNAME=
+GITHUB_REPO=
+
+KAFKA_BROKER=

.github/workflows/ci-cd.yml

@@ -0,0 +1,38 @@
+name: Knowledge Base Microservice Pipeline 
+on:
+  push:
+    branches:
+      - master
+jobs:
+  deploy-to-digital-ocean:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone the repository
+        uses: actions/checkout@v3
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_PASSWORD }}
+
+      - name: Build the image and push to Docker Hub
+        uses: docker/build-push-action@v5
+        with:
+          push: true
+          platforms: linux/amd64
+          tags: bezmoradi/knowledge-base-microservice:latest
+
+      - name: Deploy to DigitalOcean droplet
+        uses: appleboy/ssh-action@master
+        with:
+          host: ${{secrets.DIGITALOCEAN_SERVER_IPV4}}
+          username: ${{secrets.DIGITALOCEAN_SERVER_USER}}
+          key: ${{secrets.SSH_PRIVATE_KEY}}
+          script: |
+            docker stop knowledge-base-microservice
+            docker rm knowledge-base-microservice
+            docker rmi bezmoradi/knowledge-base-microservice:latest
+            docker pull bezmoradi/knowledge-base-microservice:latest
+            docker run --network my-network -d --env-file ./env-files/.env.knowledge-base --name knowledge-base-microservice bezmoradi/knowledge-base-microservice:latest
+      

.gitignore

@@ -0,0 +1,2 @@
+blog-microservices-cronjob-go
+.env

Dockerfile

@@ -0,0 +1,13 @@
+FROM golang:latest
+
+WORKDIR /app
+
+COPY go.* .
+
+RUN go mod download
+
+COPY . . 
+
+RUN go build -o binary ./src/main.go
+
+CMD ["./binary"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Bez Moradi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,109 @@
+# How to Use The Knowledge Base Microservice
+
+This project is intended to be used in tandem with the [Notification Microservice](https://github.com/bezmoradi/notification-microservice)
+
+## Intentions
+
+I use both this project and the [Notification Microservice](https://github.com/bezmoradi/notification-microservice) on a **daily** basis to review tips about the programming languages, frameworks, libraries, and especially insights from books I've previously read. By review, I mean reviewing the notes I have taken, the snippets which I have found useful, and all in all the tips which help me do my day-to-day job faster.
+
+## How Do They Work?
+
+The way that I have created this personal **Knowledge Base** is that I use a GitHub repository like [Tutorials](https://github.com/bezmoradi/tutorials) repo of mine by adding all my notes in a markdown format then it's the responsibility this repo to access [Tutorials](https://github.com/bezmoradi/tutorials) repo, fetching one document randomly then pass it to Kafka (Of course the number of documents that can be fetched is configurable).  
+Then it's the [Notification Microservice](https://github.com/bezmoradi/notification-microservice)'s job to listen to Kafka for new events and as soon as a new one is fired (Let's say every 6 hours or so), that document will be reformatted to HTML then sent to my personal email.
+
+## Installation
+
+Before installing this repo, as mentioned previously, it heavily relies on Kafka so refer to the [Notification Microservice Kafka Installation](https://github.com/bezmoradi/notification-microservice?tab=readme-ov-file#kafka-installation)'s README file to see how to install Kafka and configure it. To clone this repo, run the following command:
+
+```text
+$ git clone git@github.com:bezmoradi/knowledge-base-microservice.git
+```
+
+Make a copy of the `.env.example` file and call it `.env` which includes the following keys:
+
+```text
+MONGODB_URL=
+DATABASE=
+
+CONTENT_COUNT=
+
+GITHUB_TOKEN=
+GITHUB_USERNAME=
+GITHUB_REPO=
+
+KAFKA_BROKER=
+```
+
+To keep a record of which files of the [Tutorials](https://github.com/bezmoradi/tutorials) repo is pulled off and sent to my email, I've used MongoDB Atlas which has a free tier (This [link](https://www.mongodb.com/database/free) walks you through the process of creating a free instance of MongoDB which is more than enough for this microservice's needs).  
+After creating your Atlas database, you will be provided with a URI something similar to the following one:
+
+```text
+mongodb+srv://<username>:<password>@cluster0.z8xjp.mongodb.net/<database_name>?retryWrites=true&w=majority
+```
+
+I personally have created two databases one for development and another one for production as follows:
+
+```text
+dev_knowledge_base_microservice
+prod_knowledge_base_microservice
+```
+
+At this point, we can populate the first two keys inside the `.env` file as follows:
+
+```text
+MONGODB_URL=mongodb+srv://<username>:<password>@cluster0.z8xjp.mongodb.net/dev_knowledge_base_microservice?retryWrites=true&w=majority
+DATABASE=dev_knowledge_base_microservice
+```
+
+The `CONTENT_COUNT` key is the number of let's say tips/posts/documents you name it, we want to be sent to our email based on the CRONJOB schedule (I've set it to one for myself; meaning if the CRONJOB schedule is set to every six hours, I'll receive four tips from the [Tutorials](https://github.com/bezmoradi/tutorials) repo daily to review and expand my knowledge).
+Next is some metadata about the repository we want to pull data from:
+
+```text
+GITHUB_TOKEN=YXxTi2GmFNGE3Ga2AUZLKLB4IvgJSBw
+GITHUB_USERNAME=bezmoradi
+GITHUB_REPO=knowledge-base
+```
+
+My repository, which is private, is called `knowledge-base`. As an example, if you are interested in Go programming language, you can clone my [Go Tutorials](https://github.com/bezmoradi/tutorials/tree/master/go) repository or create your own one and call it whatever you want.  
+In order to get a GitHub Access Token, visit [Fine-grained personal access tokens](https://github.com/settings/tokens?type=beta) and create one. While creating the token, the minimum permissions this microservice needs to be able to interact with your repo is as follows:
+
+-   Read access to metadata
+-   Read and Write access to code  
+
+The finalized `.env` file should be as follows:
+
+```text
+MONGODB_URL=mongodb+srv://<username>:<password>@cluster0.z8xjp.mongodb.net/dev_knowledge_base_microservice?retryWrites=true&w=majority
+DATABASE=dev_knowledge_base_microservice
+
+CONTENT_COUNT=1
+
+GITHUB_TOKEN=YXxTi2GmFNGE3Ga2AUZLKLB4IvgJSBw
+GITHUB_USERNAME=<your_github_username>
+GITHUB_REPO=knowledge-base
+
+KAFKA_BROKER=localhost:9092
+```
+
+## How to Create A Docker Container
+
+A `Dockerfile` is also included in this repo for those (including myself) who like to run the app via Docker. First create an image:
+
+```text
+$ docker build -t knowledge-base-microservice-image
+```
+
+As the `.env` file is ignored inside the `.dockerignore` file, while creating a new container we have to pass it to docker:  
+
+```text
+$ docker run -d --env-file ./.env knowledge-base-microservice-image
+```
+
+## How to Connect The Dockerized App to Dockerized Kafka
+
+If interested following this scenario, please visit
+[How to Connect The Dockerized App to Dockerized Kafka](https://github.com/bezmoradi/notification-microservice?tab=readme-ov-file#how-to-connect-the-dockerized-app-to-dockerized-kafka) inside the Notification Microservice companion repository.
+
+## CI/CD & Deployment
+
+For setting up CI/CD and deployment, visit my walk-through in the other companion microservice at [CI/CD & Deployment](https://github.com/bezmoradi/notification-microservice?tab=readme-ov-file#cicd--deployment)

go.mod

@@ -0,0 +1,35 @@
+module github.com/behzadmoradi/blog-microservices-cronjob-go
+
+go 1.21.5
+
+require (
+	github.com/Depado/bfchroma/v2 v2.0.0
+	github.com/google/go-github v17.0.0+incompatible
+	github.com/joho/godotenv v1.5.1
+	github.com/robfig/cron/v3 v3.0.0
+	github.com/russross/blackfriday/v2 v2.1.0
+	github.com/segmentio/kafka-go v0.4.47
+	go.mongodb.org/mongo-driver v1.13.1
+	golang.org/x/oauth2 v0.15.0
+)
+
+require (
+	github.com/alecthomas/chroma/v2 v2.2.0 // indirect
+	github.com/dlclark/regexp2 v1.4.0 // indirect
+	github.com/golang/protobuf v1.5.3 // indirect
+	github.com/golang/snappy v0.0.4 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/google/go-querystring v1.1.0 // indirect
+	github.com/klauspost/compress v1.17.4 // indirect
+	github.com/montanaflynn/stats v0.7.1 // indirect
+	github.com/pierrec/lz4/v4 v4.1.19 // indirect
+	github.com/xdg-go/pbkdf2 v1.0.0 // indirect
+	github.com/xdg-go/scram v1.1.2 // indirect
+	github.com/xdg-go/stringprep v1.0.4 // indirect
+	github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a // indirect
+	golang.org/x/crypto v0.17.0 // indirect
+	golang.org/x/sync v0.5.0 // indirect
+	golang.org/x/text v0.14.0 // indirect
+	google.golang.org/appengine v1.6.8 // indirect
+	google.golang.org/protobuf v1.32.0 // indirect
+)