Sentinel API

Overview

Sentinel API is a robust, reliable webhook forwarding and delivery service built with NestJS. It is designed to ingest incoming webhooks, validate their cryptographic signatures, and reliably forward them to a target destination using a Redis-backed queue system.

By leveraging asynchronous processing and an exponential backoff retry strategy, Sentinel API ensures that temporary network failures or destination downtime do not result in data loss.

Architecture & Workflow

1. Ingestion: The system exposes a secure endpoint to receive webhook payloads.
2. Validation: Incoming requests must include a valid x-signature header.
3. Queueing: Validated payloads are immediately acknowledged (HTTP 202) and pushed to a Redis queue via Bull.
4. Processing & Delivery: A background worker processes the queue, attempting to forward the payload to the configured destination.
5. Fault Tolerance: If the delivery fails, the worker retries the operation using an exponential backoff strategy until the maximum number of attempts is reached.

---

Prerequisites

• Node.js (v18 or higher recommended)
• Docker and Docker Compose (for running Redis and database containers)
• Redis server

Installation

1. Clone the repository and navigate to the project directory.
2. Install the application dependencies:

npm install

3. Start the infrastructure using Docker Compose (provisions Redis and PostgreSQL):

docker-compose up -d

Configuration

The application relies on environment variables for configuration. Create a .env file in the root directory based on the following parameters:

# Server Configuration
PORT=3000

# Redis Configuration (Queue system)
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=

# Webhook Delivery Configuration
WEBHOOK_URL=https://example.com/webhook   # The target URL where webhooks will be forwarded
WEBHOOK_SECRET=                           # Secret key for signing webhook payloads
WEBHOOK_MAX_ATTEMPTS=5                    # Maximum delivery attempts before failing the job
WEBHOOK_BACKOFF_DELAY=10000               # Base delay in milliseconds for the exponential backoff

Running the Application

To start the application in development mode:

npm run start:dev

To build and run in production:

npm run build
npm run start

API Documentation

Receive Webhook

Ingests a webhook payload, validates the signature, and queues it for delivery.

Endpoint: POST /webhooks/receive

Headers:

• Content-Type: application/json
• x-signature: <string> (Required. Must start with sha256=)

Request Body:

{
  "event": "payment.success",
  "payload": {
    "transactionId": "txn_12345",
    "amount": 150.00,
    "currency": "USD",
    "customerId": "cus_98765"
  }
}

Responses:

• 202 Accepted The webhook was successfully received, validated, and placed in the queue.

{
  "status": "accepted"
}

• 401 Unauthorized The request is missing the signature header, or the signature format is invalid.

{
  "message": "Invalid or missing signature",
  "error": "Unauthorized",
  "statusCode": 401
}

• 400 Bad Request The request body fails validation (e.g., missing event or payload fields).

---

Testing

The project includes unit testing configured via Jest. The tests validate core logic, including controller handling and service delegation.

To run the test suite:

# Run standard tests
npm run test

# Run End-to-End (e2e) tests
npm run test:e2e

Queue Processing Details

The worker (DeliveryProcessor) runs asynchronously and picks up jobs from the delivery queue.

• It attempts an HTTP POST request to the URL defined in the WEBHOOK_URL environment variable.
• It forwards the exact event and payload received.
• It preserves the original x-signature in the outbound headers.
• On failure, Bull manages the retry cycle automatically using the configured exponential backoff variables (WEBHOOK_MAX_ATTEMPTS and WEBHOOK_BACKOFF_DELAY).