From 8cb7199064f3a3bf5833f64da2c96ba6c94bacca Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Sun, 30 Mar 2025 15:24:57 +0500 Subject: [PATCH 01/16] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 449b140..29a748b 100644 --- a/README.md +++ b/README.md @@ -61,3 +61,5 @@ The repository includes CI/CD workflows located in `.github/workflows`, which au ## Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications. + + From 107cef67ff6cf8e762fca7c20150f1e4144af069 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Mon, 31 Mar 2025 18:11:47 +0500 Subject: [PATCH 02/16] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 29a748b..f2156c0 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ # Repository Overview +--- This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. From 1b9470c4ee2c2224a21eff43752cc7409a881d53 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Wed, 2 Apr 2025 18:47:13 +0500 Subject: [PATCH 03/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f2156c0..040a489 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ The repository includes CI/CD workflows located in `.github/workflows`, which au - **Unit Testing**: Comprehensive unit tests cover all code components, ensuring high code quality and reliability. Each service is rigorously tested to validate functionality and catch potential issues early in the development cycle. - +--- ## Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications. From 0fd31bb78b4050273beaff538c062a88b68cb1b9 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Thu, 3 Apr 2025 20:18:33 +0500 Subject: [PATCH 04/16] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 040a489..491303f 100644 --- a/README.md +++ b/README.md @@ -62,5 +62,6 @@ The repository includes CI/CD workflows located in `.github/workflows`, which au --- ## Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications. +--- From 7cb8754628e17c9cf49f09a64d28c83f0b814785 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Fri, 4 Apr 2025 13:49:06 +0500 Subject: [PATCH 05/16] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 491303f..d48f56b 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,6 @@ # Repository Overview --- +--- This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. From 40396d86fda517e6a002890de56827d352845fa0 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Sat, 5 Apr 2025 14:34:31 +0500 Subject: [PATCH 06/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d48f56b..1119a0a 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Repository Overview --- ---- + This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. From dab1ca0a1ccf39edbee1d922c2700885a36e215f Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Sun, 6 Apr 2025 22:18:23 +0500 Subject: [PATCH 07/16] Update README.md --- README.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/README.md b/README.md index 1119a0a..81d0da5 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,4 @@ # Repository Overview ---- - - This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. ### API Services @@ -63,6 +60,6 @@ The repository includes CI/CD workflows located in `.github/workflows`, which au --- ## Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications. ---- + From e45f928fdf7b266aaf4c99219e364293ca3ee2fc Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Mon, 7 Apr 2025 01:12:52 +0500 Subject: [PATCH 08/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 81d0da5..e921bbd 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Repository Overview This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. - +--- ### API Services - Department Service: Exposes a **REST** API endpoint: `/api/departments/v1/departments` for retrieving department data. From d37e101302a99267a2f810ca4e724247cf213e60 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Mon, 7 Apr 2025 01:13:20 +0500 Subject: [PATCH 09/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e921bbd..81d0da5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Repository Overview This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. ---- + ### API Services - Department Service: Exposes a **REST** API endpoint: `/api/departments/v1/departments` for retrieving department data. From 4c55384fd21360ab6761ddf13f014e87c46c34ad Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Tue, 8 Apr 2025 21:38:33 +0500 Subject: [PATCH 10/16] Update README.md --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 81d0da5..9622496 100644 --- a/README.md +++ b/README.md @@ -61,5 +61,7 @@ The repository includes CI/CD workflows located in `.github/workflows`, which au ## Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications. +--- + From 3c73c3ce46837c2faa4fb372173f3139204ad736 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Wed, 9 Apr 2025 03:05:03 +0500 Subject: [PATCH 11/16] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 9622496..5069ff8 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,7 @@ The repository includes CI/CD workflows located in `.github/workflows`, which au --- ## Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications. +--- --- From a758792bd607c3a834c7b55e17694cc610ea23fa Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Thu, 10 Apr 2025 01:39:23 +0500 Subject: [PATCH 12/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5069ff8..0750940 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ This repository showcases a robust microservices architecture comprising three d This design promotes modularity and scalability across the services. ## Key Features - + ### Architectural Patterns & Design Choices * **Concurrency Pattern:** * Utilized in [service/user_service/user/user_service](https://github.com/syedomair/backend-microservices/blob/main/service/user_service/user/user_serivce.go) to execute multiple database queries and gRPC calls concurrently using Go's `errgroup`. From 4fc1bf7d2b303499fd1c2c94d4e682806f9d3273 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Fri, 11 Apr 2025 07:03:16 +0500 Subject: [PATCH 13/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0750940..a7375ed 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Repository Overview This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. - +--- ### API Services - Department Service: Exposes a **REST** API endpoint: `/api/departments/v1/departments` for retrieving department data. From 701880c90bc1a39c913234bfffb3b086f1d1def4 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Sat, 12 Apr 2025 00:45:44 +0500 Subject: [PATCH 14/16] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a7375ed..0750940 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Repository Overview This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. ---- + ### API Services - Department Service: Exposes a **REST** API endpoint: `/api/departments/v1/departments` for retrieving department data. From 070cb2d5ba21d497e41cfd9003f64b08aec02687 Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Sun, 13 Apr 2025 08:28:29 +0500 Subject: [PATCH 15/16] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 0750940..5c782a2 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # Repository Overview This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. + ### API Services - Department Service: Exposes a **REST** API endpoint: `/api/departments/v1/departments` for retrieving department data. From 292f88fa3c62a5c8e2c37596683d7fa4842711fc Mon Sep 17 00:00:00 2001 From: Syed Omair Date: Wed, 8 Oct 2025 11:47:31 +0500 Subject: [PATCH 16/16] Readme updated with diagram --- README.md | 74 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 48 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 449b140..55b88b5 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,43 @@ -# Repository Overview +# Distributed Microservices Demo in Go -This repository showcases a robust microservices architecture comprising three distinct services: **Department**, **User**, and **Point**. The project emphasizes code reusability through a common container that initializes essential components such as logging, database connections, and environment variables. +A cloud-native, production-ready demonstration of a distributed system built with Go. This project features three containerized microservices (REST & gRPC) communicating over a network, backed by PostgreSQL, deployed via CI/CD to AWS ECS, and fully monitored with Prometheus and Grafana. -### API Services -- Department Service: - Exposes a **REST** API endpoint: `/api/departments/v1/departments` for retrieving department data. -- User Service: - Provides a **REST** API endpoint: `/api/users/v1/users` for accessing user information. -- Point Service: - Utilizes **gRPC** to deliver user point data, which is consumed by the User service, effectively demonstrating inter-service communication. +# Demo +**Run Locally:** +```bash +make run_docker +``` +Prerequisites: Docker and Docker Compose must be installed on your machine. -This design promotes modularity and scalability across the services. -## Key Features +# Architecture Overview -### Architectural Patterns & Design Choices +```mermaid +graph TB + subgraph AWS ECS / Docker Compose + A[Client] --> B[API Gateway] + B --> C[User Service] + C --> E[(User DB)] + C -- gRPC --> G[Point Service] + G --> H[(Point DB)] + C --> I[Prometheus] + D --> I[Prometheus] + G --> I[Prometheus] + I --> J[Grafana] + B --> D[Department Service] + D --> F[(Department DB)] + end +``` + +# Technology Stack +Go, PostgreSQL, Docker, gRPC, Prometheus, Grafana, GitHub Action + +# Key Features + +### ๐Ÿ—๏ธ Architecture & Design Patterns +This project is a textbook example of practical software engineering patterns in Go: * **Concurrency Pattern:** - * Utilized in [service/user_service/user/user_service](https://github.com/syedomair/backend-microservices/blob/main/service/user_service/user/user_serivce.go) to execute multiple database queries and gRPC calls concurrently using Go's `errgroup`. + * Utilized in [service/user_service/user/user_service](https://github.com/syedomair/backend-microservices/blob/main/service/user_service/user/user_service.go) to execute multiple database queries and gRPC calls concurrently using Go's `errgroup`. * Enhances the performance of the `GetAllUserStatistics` method by leveraging parallel processing. * **Dependency Injection Pattern:** * Utilized in [lib/container/container.go](https://github.com/syedomair/backend-microservices/blob/main/lib/container/container.go) to manage logging, database connections, and environment variables. @@ -43,21 +64,22 @@ This design promotes modularity and scalability across the services. * Implemented in [lib/container/connection.go](https://github.com/syedomair/backend-microservices/blob/main/lib/container/connection.go) to manage a pool of reusable gRPC client connections. * Optimizes resource usage and improves performance by reducing the overhead of repeatedly creating and destroying connections. -### CI/CD Integration: -The repository includes CI/CD workflows located in `.github/workflows`, which automate the deployment process to AWS Elastic Container Registry (ECR) and Elastic Container Service (ECS) servers. This ensures seamless updates and efficient management of service deployments. +### ๐Ÿš€ Operational Excellence +* **CI/CD:** Automated Docker image builds and deployments to AWS ECS via GitHub Actions. +* **Monitoring:** Integrated Prometheus metrics and pprof profiling for real-time performance insight. +* **Observability:** Structured logging and request tracing throughout the services. +* **Containerization:** Fully dockerized for local development and cloud deployment. + -### Performance Monitoring -- **Prometheus Metrics**: Integrated Prometheus metrics allow users to monitor the performance of each service in real-time. This feature provides insights into system health and resource utilization. - -- **Memory Profiling with pprof**: - The project includes pprof for memory monitoring, enabling developers to analyze memory usage and optimize performance effectively. +### ๐Ÿงช Testing Strategy +* **Unit Tests:** Comprehensive tests for all business logic and handlers. +* **Integration Tests:** End-to-end tests using a live test database and gRPC server within Docker, validating the entire service ecosystem. -### Testing Framework -- **Integration Testing**: - The system performs integration testing using a mock database running in a test Docker container. This setup ensures that all services interact correctly and maintain data integrity during operations. +### ๐Ÿ“ก APIs & Communication +* **RESTful APIs:** JSON over HTTP for `user-service` (`/users`) and `department-service` (`/departments`). +* **gRPC:** High-performance RPC for internal communication between `user-service` and `point-service`. -- **Unit Testing**: - Comprehensive unit tests cover all code components, ensuring high code quality and reliability. Each service is rigorously tested to validate functionality and catch potential issues early in the development cycle. +--- -## Conclusion +# Conclusion This microservices architecture not only demonstrates best practices in software design but also incorporates essential features for modern application development, such as CI/CD, performance monitoring, and robust testing frameworks. By leveraging these technologies, developers can build scalable, maintainable, and high-performing applications.