Skip to content

Tech Stack Report

Jump to bottom
Morales Patty Jose Manuel edited this page Oct 27, 2024 · 1 revision

Tech Stack Report

Table of Contents

Introduction

This technical report outlines the high-level architecture of our software system, detailing each component, its responsibilities, technologies used, and interactions within the ecosystem. The architecture is designed to ensure scalability, maintainability, security, and optimal performance, leveraging modern frameworks and best practices.

System Architecture Overview

The system comprises a frontend built with Next.js, an API Gateway using Ocelot, a service registry and discovery mechanism with Consul, consolidated microservices handling distinct business capabilities, and supporting infrastructure components such as a message bus, logging service, and monitoring tools. The entire ecosystem is containerized using Docker and managed through CI/CD pipelines for streamlined deployments.

1. Frontend (Next.js)

  • Framework: Next.js (React)
  • Responsibilities:
    • User Interface (UI): Provides an interactive and responsive interface for users.
    • Server-Side Rendering (SSR): Enhances performance and SEO by rendering pages on the server.
    • Routing: Manages both dynamic and static routes for seamless navigation.
    • SEO Optimization: Implements best practices to improve search engine visibility.
    • API Integration: Communicates with backend services via RESTful APIs.
    • Authentication: Integrates directly with Firebase Authentication for user management and authentication.
  • Deployment: Vercel
    • Features:
      • Automatic deployments on every push.
      • Preview deployments for pull requests.
      • Optimized performance and global CDN distribution.

2. API Gateway (Ocelot)

  • Technology: Ocelot running on ASP.NET Core
  • Responsibilities:
    • Single Entry Point: Acts as the main access point for all client requests.
    • Request Routing: Directs incoming HTTP requests to the appropriate microservices.
    • Authentication Validation: Validates JWT tokens issued by Firebase Authentication.
    • Rate Limiting: Implements rate limiting to prevent API abuse.
    • Retry Policies: Utilizes Polly for implementing retry policies and circuit breakers.
  • Integration:
    • Service Discovery: Connects with Consul to dynamically discover service instances.

3. Service Registry and Discovery (Consul)

  • Technology: Consul
  • Responsibilities:
    • Dynamic Service Registration: Automatically registers microservices as they become available.
    • Service Discovery: Enables services to locate each other without hard-coded addresses.
    • Health Monitoring: Performs health checks to ensure services are operational.
    • Configuration Storage: Maintains key-value pairs for distributed configuration management.

4. Microservices

Shared Architecture

All microservices share a common architectural foundation to ensure consistency, scalability, and maintainability.

Individual Services

  1. Users & Jobs Service

    • Functions:
      • User registration, authentication, and profile management
      • Role-based access control (RBAC)
      • Job posting and management
      • Job search with advanced filters
      • Application tracking
    • Database: PostgreSQL

  2. Communication Service

    • Functions:
      • Real-time messaging using WebSockets
      • Push notifications for mobile and desktop
      • Email notifications
      • Message and notification history management
    • Database: PostgreSQL
    • Additional Technologies:

  3. AI Service

    • Functions:
      • AI-based CV and profile suggestions
      • Content moderation for job postings and user-generated content
      • Skill recommendation based on market trends
    • Database: PostgreSQL
    • Additional Technologies:
      • Integration with OpenAI API or similar language models
      • ML.NET for custom machine learning models

5. Message Bus (RabbitMQ)

  • Technology: RabbitMQ
  • Function: Facilitates asynchronous communication between microservices through message queues, enabling loose coupling, improved scalability, and enhanced fault tolerance.

6. Logging Service (Seq)

  • Technology: Seq
  • Function: Centralizes and visualizes structured logs from all microservices, allowing for real-time log analysis, troubleshooting, and monitoring.

7. Monitoring and Observability (OpenTelemetry)

  • Technology: OpenTelemetry
  • Functions:
    • Metrics Collection: Gathers performance metrics from various components.
    • Distributed Tracing: Tracks the flow of requests across microservices to identify bottlenecks and performance issues.
    • Data Export: Sends collected data to observability backends for analysis and visualization.
  • Integration: Grafana and Prometheus for metrics visualization and alerting.

8. Databases

  • PostgreSQL:
    • Usage: Primary database for all microservices.
    • Advantages: Robust support for complex queries, transactions, and data integrity.
  • Redis:
    • Usage: Distributed caching to improve performance and reduce database load.
    • Functions: Session storage, request caching, and distributed locking.

9. Containerization (Docker)

  • Technology: Docker
  • Function: Packages microservices and infrastructure components into containers, ensuring consistency across development, testing, and production environments. Facilitates scalable and efficient deployments.
  • Orchestration: Kubernetes for container orchestration and management in production.

10. CI/CD and Deployment

  • Continuous Integration/Continuous Deployment (CI/CD):
    • Tool: GitHub Actions
    • Functions:
      • Automates build, test, and deployment pipelines.
      • Ensures code quality through automated testing and static code analysis.
      • Enables rapid and reliable deployments.
  • Deployment Targets:
    • Backend: Render handles the deployment of microservices, offering features like auto-scaling and zero-downtime deployments.
    • Frontend: Vercel manages the deployment of the Next.js frontend, providing seamless integration with GitHub and instant rollbacks.

11. Additional Technical Considerations

Data Persistence

  • Entity Framework Core:
    • Utilized as the ORM for interacting with PostgreSQL databases.
    • Implements the repository and unit of work patterns for data access abstraction.
    • Manages database migrations to maintain schema versioning and consistency.
  • Database Optimization:
    • Implements indexes tailored to each microservice's query patterns.
    • Employs query optimization techniques to enhance performance.
    • Utilizes database partitioning for large-scale data management.

Inter-Service Communication

  • Asynchronous Communication:
    • Managed by RabbitMQ, allowing microservices to communicate via message queues without tight coupling.
    • Implements the outbox pattern for ensuring message delivery and maintaining data consistency across services.
  • Synchronous Communication:
    • Handled through HTTP calls facilitated by the Ocelot API Gateway, enabling real-time interactions.
    • Implements retry policies and circuit breakers using Polly to enhance resilience.

Security

  • Authentication:
    • Managed via Firebase Authentication, issuing and validating JWT tokens to secure API endpoints.
    • Implements refresh token rotation for enhanced security.
  • Data Protection:
    • All communications are secured using HTTPS/TLS to ensure data confidentiality and integrity.
    • Sensitive data within databases is encrypted at rest and in transit.
    • Implements data masking for sensitive information in logs and error messages.
  • API Security:
    • Implements rate limiting and throttling to prevent abuse.
    • Uses OAuth 2.0 and OpenID Connect for secure authorization flows.

Scalability

  • Horizontal Scaling:
    • Microservices are designed to be stateless, facilitating horizontal scaling by adding or removing instances based on demand.
    • Utilizes Kubernetes horizontal pod autoscaler for automatic scaling based on CPU and memory metrics.
  • Database Scaling:
    • Implements read replicas for PostgreSQL to distribute read traffic.
    • Utilizes database sharding for high-volume data services.

API Documentation

  • OpenAPI (Swagger):
    • Each microservice utilizes OpenAPI to generate interactive and up-to-date API documentation.
    • Implements API versioning to manage changes and ensure backward compatibility.

Error Handling and Resilience

  • Centralized Error Handling:
    • Implements a global exception handling middleware in each microservice.
    • Utilizes Polly for implementing retry policies, circuit breakers, and bulkheads.

Performance Optimization

  • Caching Strategy:
    • Implements a multi-level caching strategy using Redis and in-memory caching.
    • Utilizes cache-aside and write-through patterns for data consistency.
  • Asynchronous Processing:
    • Leverages asynchronous programming models in ASP.NET Core for improved throughput.

Conclusion

The updated architecture leverages modern technologies and best practices to build a scalable, maintainable, and secure system. By consolidating microservices and improving the overall structure, we've enhanced system efficiency while maintaining modularity. The direct integration of Firebase Authentication with the frontend simplifies the authentication flow. The implementation of advanced caching strategies, robust security measures, and comprehensive monitoring ensures the system remains performant, secure, and observable as it scales to meet future demands.

References

Clone this wiki locally