Microservice Series - From Zero to Hero

This repo is used for my 65 blog post series "Microservices - From Zero to Hero". It started out as .NET Core 3.1 project and has been upgraded to .NET 6 since then. The following tools and technologies are used:

• .NET 6
• Swagger
• Asynchronous messaging using RabbitMQ and Azure Service Bus
• Azure Functions
• CI/CD YAML pipelines
• Docker
• Helm
• Kubernetes
• KEDA
• AAD authentication

For more details, see the corresponding blog posts.

Getting Started with Microservices

The following posts explain the theory behind microservices and how to set up your first two .NET 6 (originally .NET Core 3.1) microservices. Both microservices use the mediator pattern and communicate via RabbitMQ.

• Microservices - Getting Started

• Programming a Microservice with .NET Core 3.1

• Document your Microservice with Swagger

• CQRS in ASP .NET Core 3.1

• Mediator Pattern in ASP .NET Core 3.1

• RabbitMQ in an ASP .NET Core 3.1 Microservice

• Dockerize an ASP .NET Core Microservice and RabbitMQ

• Set up Docker-Compose for ASP .NET Core 3.1 Microservices

• Upgrade a Microservice from .NET Core 3.1 to .NET 5.0

• Upgrade a Microservice from .NET 5.0 to .NET 6.0

Continuous Integration and Unit Tests in Azure DevOps

This section is all about automated builds using YAML pipelines in Azure DevOps. Starting with a simple .NET Core pipeline, the pipeline switches to building Docker images and then runs xUnit tests inside the Docker container. Another topic of this section is automated versioning of Docker images and lastly splitting up the pipeline into smaller chunks using templates.

• Build .NET Core in a CI Pipeline in Azure DevOps

• Run the CI Pipeline during a Pull Request

• Build Docker in an Azure DevOps CI Pipeline

• Run xUnit Tests inside Docker during an Azure DevOps CI Build

• Get xUnit Code Coverage from Docker

• Automatically Version Docker Containers in Azure DevOps CI

• Improve Azure DevOps YAML Pipelines with Templates

• Split up the CI/CD Pipeline into two Pipelines

Continuous Deployment with Azure DevOps

After building the Docker images, let's focus on deploying these images to Kubernetes. Each pull request gets deployed into a new namespace, SSL certificates, and a unique URL is generated and Helm is used to manage the deployment. Using Helm allows overriding configuration values. Furthermore, this section explains how to deploy Azure Functions, and SQL Databases and push NuGet packages to an internal or public feed.

• Approvals for YAML Pipelines in Azure DevOps

• Deploy to Azure Kubernetes Service using Azure DevOps YAML Pipelines

• Improve Azure DevOps YAML Pipelines with Templates

• Replace Helm Chart Variables in your CI/CD Pipeline with Tokenizer

• Split up the CI/CD Pipeline into two Pipelines

• Deploy Microservices to multiple Environments using Azure DevOps

• Deploy every Pull Request into a dedicated Namespace in Kubernetes

• Automatically set Azure Service Bus Queue Connection Strings during the Deployment

• Deploy Azure Functions with Azure DevOps YAML Pipelines

• Deploy a Docker Container to Azure Functions using an Azure DevOps YAML Pipeline

• Publish NuGet Packages to Nuget.org using Azure DevOps Pipelines

• Automatically Deploy your Database with Dacpac Packages using Linux and Azure DevOps

• Deploy KEDA and an Autoscaler using Azure DevOps Pipelines

• Create Custom Roles for Azure DevOps in Azure

• Update DNS Records in an Azure DevOps Pipeline

Kubernetes with Helm

The following posts explain Microsoft's Azure Kubernetes Service and why Helm is useful for your deployments. After the basics, more advanced topics like Ingress controller, automated SSL certificate installation, and KEDA are discussed.

• Azure Kubernetes Service - Getting Started

• Helm - Getting Started

• Deploy to Kubernetes using Helm Charts

• Override Appsettings in Kubernetes

• Run a Kubernetes Cluster locally

• Manage Resources in Kubernetes

• Auto-scale in Kubernetes using the Horizontal Pod Autoscaler

• Readiness and Liveness Probes in Kubernetes

• Use Azure Container Registry in Kubernetes

• Automatically scale your AKS Cluster

• Debug Microservices running inside a Kubernetes Cluster with Bridge to Kubernetes

• Set up Nginx as Ingress Controller in Kubernetes

• Configure custom URLs to access Microservices running in Kubernetes

• Automatically issue SSL Certificates and use SSL Termination in Kubernetes

• KEDA - Kubernetes Event-driven Autoscaling

SSL Configuration in Kubernetes

Kubernetes helps to automatically create Let's Encrypt SSL certificates and Nginx as Ingress controller allows the creation of unique URLs for each microservice.

• Set up Nginx as Ingress Controller in Kubernetes

• Configure custom URLs to access Microservices running in Kubernetes

• Automatically issue SSL Certificates and use SSL Termination in Kubernetes

Create NuGet Packages

NuGet packages allow sharing of code between microservices. Additionally, the versioning of these packages gives developers full control over the version they are using and when they want to upgrade to newer versions.

• Create NuGet Packages in Azure DevOps Pipelines

• Publish to an Internal NuGet Feed in Azure DevOps

• Restore NuGet Packages from a Private Feed when building Docker Containers

• Publish NuGet Packages to Nuget.org using Azure DevOps Pipelines

Database Deployments with Azure DevOps

Deploying database changes has always been a pain. Using dacpac packages allows developers or database administrators to easily deploy their changes to an existing database, or create a new one with a pre-defined schema and optionally with test data. Docker also comes in handy when trying to deploy the dacpac package using Linux. Since Azure DevOps doesn't support deploying dacpacs on Linux, a custom Docker container is used to deploy the dacpac.

• Automate Database Deployments

• Automatically deploy Database Changes with SSDT

• Automatically Deploy your Database with Dacpac Packages using Linux and Azure DevOps

• Use a Database with a Microservice running in Kubernetes

Azure Container Registry and Azure Service Bus

The Azure Container Registry is a private repository in Azure. Since it is private, Kubernetes needs an Image pull secret to be able to download images from there. Additionally, this section shows how to replace RabbitMQ with Azure Service Bus Queues and how to replace the .NET background process with Azure Functions to process messages in these queues.

• Use Azure Container Registry in Kubernetes

• Replace RabbitMQ with Azure Service Bus Queues

• Use Azure Functions to Process Queue Messages

Azure Functions

Azure Functions can be used to process messages from queues and can be deployed as a Docker container or as .NET 6 application.

• Deploy Azure Functions with Azure DevOps YAML Pipelines

• Deploy a Docker Container to Azure Functions using an Azure DevOps YAML Pipeline

Infrastructure as Code, Monitoring, and Logging

Infrastructure as Code (IaC) allows developers to define the infrastructure and all its dependencies as code. These configurations are often stored in YAML files and have the advantage that they can be checked into version control and also can be deployed quickly using Azure DevOps. Another aspect of operating a Kubernetes infrastructure is logging and monitoring with tools such as Loki or Prometheus.

• Use Infrastructure as Code to deploy your Infrastructure with Azure DevOps

• Collect and Query your Kubernetes Cluster Logs with Grafana Loki

• Monitor .NET Microservices in Kubernetes with Prometheus

• Create Grafana Dashboards with Prometheus Metrics

Service Mesh

Big Kubernetes clusters can be hard to manage. Service Mesh like Istio helps administrators to manage Kubernetes clusters with topics such as SSL connections, monitoring, or tracing. All that can be achieved without any changes to the existing applications. Isitio also comes with a bunch of add-ons such as Grafana, Kiali, and Jaeger to help administrate the cluster.

• Service Mesh in Kubernetes - Getting Started

• Istio in Kubernetes - Getting Started

• Use Istio to manage your Microservices

• Add Istio to an existing Microservice in Kubernetes

KEDA - Kubernetes Event-Driven Autoscaling

Applications have become more and more complicated over the years and often rely on external dependencies these days. These dependencies could be an Azure Service Bus Queue or a database. KEDA allows applications to scale according to these dependencies.

• KEDA - Kubernetes Event-driven Autoscaling

• Deploy KEDA and an Autoscaler using Azure DevOps Pipelines

AAD Authentication

Using Azure Active Directory authentication allows users to authenticate their applications using Azure identities. The advantage of this approach is that no passwords need to be stored and managed for the connection.

• Use AAD Authentication for Pods running in AKS

• Implement AAD Authentication to access Azure SQL Databases

• Use AAD Authentication for Applications running in AKS to access Azure SQL Databases

All Posts in Chronological Order

The following list consists of all blog posts in chronological order:

• Microservices - Getting Started

• Programming a Microservice with .NET Core 3.1

• Document your Microservice with Swagger

• CQRS in ASP .NET Core 3.1

• Mediator Pattern in ASP .NET Core 3.1

• RabbitMQ in an ASP .NET Core 3.1 Microservice

• Dockerize an ASP .NET Core Microservice and RabbitMQ

• Set up Docker-Compose for ASP .NET Core 3.1 Microservices

• Build .NET Core in a CI Pipeline in Azure DevOps

• Run the CI Pipeline during a Pull Request

• Build Docker in an Azure DevOps CI Pipeline

• Run xUnit Tests inside Docker during an Azure DevOps CI Build

• Upgrade a Microservice from .NET Core 3.1 to .NET 5.0

• Get xUnit Code Coverage from Docker

• Azure Kubernetes Service - Getting Started

• Helm - Getting Started

• Deploy to Kubernetes using Helm Charts

• Override Appsettings in Kubernetes

• Run a Kubernetes Cluster locally

• Automatically Version Docker Containers in Azure DevOps CI

• Create NuGet Packages in Azure DevOps Pipelines

• Publish to an Internal NuGet Feed in Azure DevOps

• Restore NuGet Packages from a Private Feed when building Docker Containers

• Publish NuGet Packages to Nuget.org using Azure DevOps Pipelines

• Approvals for YAML Pipelines in Azure DevOps

• Deploy to Azure Kubernetes Service using Azure DevOps YAML Pipelines

• Improve Azure DevOps YAML Pipelines with Templates

• Manage Resources in Kubernetes

• Auto-scale in Kubernetes using the Horizontal Pod Autoscaler

• Replace Helm Chart Variables in your CI/CD Pipeline with Tokenizer

• Automate Database Deployments

• Automatically deploy Database Changes with SSDT

• Automatically Deploy your Database with Dacpac Packages using Linux and Azure DevOps

• Use a Database with a Microservice running in Kubernetes

• Readiness and Liveness Probes in Kubernetes

• Use Azure Container Registry in Kubernetes

• Replace RabbitMQ with Azure Service Bus Queues

• Use Azure Functions to Process Queue Messages

• Deploy Azure Functions with Azure DevOps YAML Pipelines

• Deploy a Docker Container to Azure Functions using an Azure DevOps YAML Pipeline

• Set up Nginx as Ingress Controller in Kubernetes

• Configure custom URLs to access Microservices running in Kubernetes

• Automatically issue SSL Certificates and use SSL Termination in Kubernetes

• Split up the CI/CD Pipeline into two Pipelines

• Deploy Microservices to multiple Environments using Azure DevOps

• Deploy every Pull Request into a dedicated Namespace in Kubernetes

• Use Infrastructure as Code to deploy your Infrastructure with Azure DevOps

• Debug Microservices running inside a Kubernetes Cluster with Bridge to Kubernetes

• Collect and Query your Kubernetes Cluster Logs with Grafana Loki

• Monitor .NET Microservices in Kubernetes with Prometheus

• Create Grafana Dashboards with Prometheus Metrics

• Service Mesh in Kubernetes - Getting Started

• Istio in Kubernetes - Getting Started

• Use Istio to manage your Microservices

• Add Istio to an existing Microservice in Kubernetes

• KEDA - Kubernetes Event-driven Autoscaling

• Deploy KEDA and an Autoscaler using Azure DevOps Pipelines

• Create Custom Roles for Azure DevOps in Azure

• Update DNS Records in an Azure DevOps Pipeline

• Automatically scale your AKS Cluster

• Use AAD Authentication for Pods running in AKS

• Implement AAD Authentication to access Azure SQL Databases

• Automatically set Azure Service Bus Queue Connection Strings during the Deployment

• Upgrade a Microservice from .NET 5.0 to .NET 6.0

• Use AAD Authentication for Applications running in AKS to access Azure SQL Databases