Skip to content

JDGrillo/durable-functions-testing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

2 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Azure Durable Functions Load Testing Application

A comprehensive Node.js application built with Azure Durable Functions, designed for demonstrating and load testing stateful serverless workflows. Features orchestrations, activities, durable entities, and dual storage backends.

🎯 Overview

This application showcases Azure Durable Functions patterns for load testing scenarios:

  • HTTP-Triggered Orchestrations - Fan-out/fan-in and sequential workflow patterns
  • Session Management - CRUD operations with Azure Table Storage or Cosmos DB
  • Durable Entities - Stateful metrics tracking and session counters
  • Activity Functions - Parallel and sequential task processing
  • Application Insights - Full observability with custom telemetry
  • Authentication - JWT validation with Azure Entra ID (optional bypass for testing)

πŸ—οΈ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     Azure Durable Functions                     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                                                                 β”‚
β”‚  HTTP APIs          Orchestrators        Activities            β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”‚
β”‚  β”‚ Sessions │─────▢│ Workflow     │───▢│ ProcessItem  β”‚       β”‚
β”‚  β”‚  - POST  β”‚      β”‚ Orchestrator β”‚    β”‚ Aggregate    β”‚       β”‚
β”‚  β”‚  - GET   β”‚      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β”‚ UpdateMetricsβ”‚       β”‚
β”‚  β”‚  - DELETEβ”‚                           β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                           β”‚
β”‚                    β”‚ Fan-Out/In   β”‚    Durable Entities        β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”      β”‚ Orchestrator β”‚    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”‚
β”‚  β”‚Orchestrateβ”‚β”€β”€β”€β”€β–Άβ””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β”‚ Metrics      β”‚       β”‚
β”‚  β”‚  - POST  β”‚                          β”‚ Counter      β”‚       β”‚
β”‚  β”‚  - GET   β”‚                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                                  β”‚
β”‚                                                                 β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                      Storage Layer                              β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       β”‚
β”‚  β”‚ Azure Table Storage    β”‚  β”‚ Azure Cosmos DB        β”‚       β”‚
β”‚  β”‚ (Default - Dev/Test)   β”‚  β”‚ (Optional - Production)β”‚       β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
                              β–Ό
                 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                 β”‚ Application Insights  β”‚
                 β”‚ - Telemetry           β”‚
                 β”‚ - Custom Metrics      β”‚
                 β”‚ - Distributed Tracing β”‚
                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ“‹ Prerequisites

Before starting, ensure you have:

Required

  • Node.js v20.17.0 or higher (v22+ recommended)
  • npm v10.8.2 or higher
  • Azure Functions Core Tools v4.x - Install Guide
  • Azurite (Azure Storage Emulator) - Install Guide

Optional (for Azure Deployment)

Verify Installation

node --version          # Should be v20.17.0 or higher
npm --version           # Should be v10.8.2 or higher
func --version          # Should be 4.x
azurite --version       # Should be 3.x

πŸš€ Quick Start

1. Clone and Install Dependencies

git clone <repository-url>
cd node-durable
npm install

2. Configure Local Settings

cp local.settings.json.example local.settings.json

Edit local.settings.json if needed (defaults work for local development):

{
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "node",
    "STORAGE_TYPE": "tables",
    "BYPASS_AUTH": "true"
  }
}

3. Start Azurite (Storage Emulator)

Option A - Separate Terminal:

azurite --silent

Option B - Background (Windows PowerShell):

Start-Process -NoNewWindow azurite -ArgumentList "--silent"

Option C - NPX (one-time):

npx azurite --silent

4. Build and Start the Application

npm run build
npm start

You should see output like:

Azure Functions Core Tools
Core Tools Version:       4.0.5907 Commit hash: N/A +591b8aec842e333a87ea9e23ba390bb5effe0655 (64-bit)
Function Runtime Version: 4.28.5.21810

Functions:

        createSession: [POST] http://localhost:7071/api/sessions

        deleteSession: [DELETE] http://localhost:7071/api/sessions/{id}

        getSession: [GET] http://localhost:7071/api/sessions/{id}

        orchestrateFanOut: [POST] http://localhost:7071/api/orchestrate/fanout

        orchestrateStart: [POST] http://localhost:7071/api/orchestrate

        orchestrateStatus: [GET] http://localhost:7071/api/orchestrate/{instanceId}

For detailed output, run func with --verbose flag.

5. Test the API

Test Session Creation:

curl -X POST http://localhost:7071/api/sessions \
  -H "Content-Type: application/json" \
  -d '{"userId": "user123", "data": {"name": "Test User"}}'

Expected response (HTTP 201):

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "userId": "user123",
  "data": { "name": "Test User" },
  "createdAt": "2026-03-25T12:00:00.000Z",
  "lastAccessedAt": "2026-03-25T12:00:00.000Z"
}

Test Session Retrieval:

curl http://localhost:7071/api/sessions/{sessionId}

Test Orchestration (Note: Known issue with durable client binding):

curl -X POST http://localhost:7071/api/orchestrate \
  -H "Content-Type: application/json" \
  -d '{"items": ["item1", "item2"], "userId": "user123"}'

πŸ“š API Documentation

Session Management

Create Session

POST /api/sessions
Content-Type: application/json

{
  "userId": "string (required)",
  "data": {
    // Any JSON object (optional)
  }
}

Response (201 Created):

{
  "id": "uuid",
  "userId": "string",
  "data": {},
  "createdAt": "ISO8601",
  "lastAccessedAt": "ISO8601"
}

Get Session

GET /api/sessions/{id}

Response (200 OK):

{
  "id": "uuid",
  "userId": "string",
  "data": {},
  "createdAt": "ISO8601",
  "lastAccessedAt": "ISO8601"
}

Delete Session

DELETE /api/sessions/{id}

Response (200 OK):

{
  "message": "Session deleted successfully"
}

Orchestration Endpoints

Start Workflow Orchestration

POST /api/orchestrate
Content-Type: application/json

{
  "items": ["string"],
  "userId": "string"
}

Response (202 Accepted):

{
  "instanceId": "uuid",
  "statusQueryGetUri": "http://...",
  "sendEventPostUri": "http://...",
  "terminatePostUri": "http://...",
  "purgeHistoryDeleteUri": "http://..."
}

Get Orchestration Status

GET /api/orchestrate/{instanceId}

Response (200 OK):

{
  "name": "processWorkflowOrchestrator",
  "instanceId": "uuid",
  "runtimeStatus": "Completed|Running|Failed",
  "input": {},
  "output": {},
  "createdTime": "ISO8601",
  "lastUpdatedTime": "ISO8601"
}

Start Fan-Out/Fan-In Orchestration

POST /api/orchestrate/fanout
Content-Type: application/json

{
  "items": ["string"],
  "userId": "string"
}

πŸ”§ Configuration

Environment Variables

Variable Description Default Required
AzureWebJobsStorage Storage connection string UseDevelopmentStorage=true Yes
FUNCTIONS_WORKER_RUNTIME Runtime environment node Yes
FUNCTIONS_NODE_VERSION Node.js version 22 No
STORAGE_TYPE Storage backend (tables or cosmos) tables Yes
STORAGE_ACCOUNT_NAME Azure Storage account name - If using Azure
COSMOS_ENDPOINT Cosmos DB endpoint - If STORAGE_TYPE=cosmos
BYPASS_AUTH Skip authentication for testing true No
ENTRA_TENANT_ID Azure Entra ID tenant - If auth enabled
ENTRA_CLIENT_ID Azure Entra ID app client ID - If auth enabled
APPLICATIONINSIGHTS_CONNECTION_STRING Application Insights connection - No

Storage Backend Options

Azure Table Storage (Default - Recommended for Development)

  • Pros: Lower cost, simpler setup, sufficient for most scenarios
  • Cons: Limited query capabilities, single-region
  • Configuration: Set STORAGE_TYPE=tables

Azure Cosmos DB (Optional - Recommended for Production Scale)

  • Pros: Global distribution, better performance at scale, advanced queries
  • Cons: Higher cost, more complexity
  • Configuration:
    • Set STORAGE_TYPE=cosmos
    • Set COSMOS_ENDPOINT to your Cosmos DB endpoint
    • Enable in Terraform: enable_cosmos_db = true

πŸ§ͺ Testing

Manual Testing with cURL

See API Documentation above for endpoint examples.

Testing Checklist

  • Azurite Running - Check http://localhost:10000/ accessible
  • Functions Running - Check logs show "Worker process started and initialized"
  • Create Session - POST to /api/sessions returns 201
  • Get Session - GET to /api/sessions/{id} returns 200
  • Delete Session - DELETE to /api/sessions/{id} returns 200
  • Start Orchestration - POST to /api/orchestrate returns 202
  • Check Status - GET orchestration status
  • Fan-Out Pattern - POST to /api/orchestrate/fanout

Known Issues

Local Tooling Assembly Error (Non-blocking)

Problem: On some machines, func start may fail with System.Net.Http, Version=8.0.0.0 assembly not found

Impact: Prevents local startup. Does not affect Azure deployments.

Fix: Update Azure Functions Core Tools to the latest version:

npm install -g azure-functions-core-tools@4 --unsafe-perm true

πŸ“Š Application Insights Integration

The application includes comprehensive Application Insights telemetry:

Auto-Collected

  • HTTP requests and responses
  • Dependencies (Storage, Cosmos DB calls)
  • Performance counters
  • Exceptions and errors
  • Console logs
  • Heartbeat metrics

Custom Telemetry

  • Events: SessionCreated, SessionRetrieved
  • Metrics: SessionsCreated counter
  • Exceptions: Operation-specific error tracking

Local Development

Application Insights gracefully skips telemetry when APPLICATIONINSIGHTS_CONNECTION_STRING is empty. Set the connection string to enable telemetry during local testing.

Viewing Telemetry

Once deployed to Azure:

  1. Navigate to Application Insights in Azure Portal
  2. View Live Metrics for real-time telemetry
  3. Query Logs with KQL for detailed analysis
  4. Check Performance for request insights

πŸš€ Azure Deployment

Using Terraform (Recommended)

See detailed instructions in infra/README.md.

Quick Deploy:

cd infra

# Configure variables
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values

# Initialize and deploy
terraform init
terraform plan
terraform apply

# Deploy application code
cd ..
npm run build
func azure functionapp publish $(terraform -chdir=infra output -raw function_app_name)

Using Azure CLI

# Login to Azure
az login

# Create resource group
az group create --name rg-durable-functions --location eastus

# Create storage account
az storage account create \
  --name stdurabledev \
  --resource-group rg-durable-functions \
  --location eastus \
  --sku Standard_LRS

# Create function app
az functionapp create \
  --name func-durable-loadtest \
  --resource-group rg-durable-functions \
  --storage-account stdurabledev \
  --consumption-plan-location eastus \
  --runtime node \
  --runtime-version 20 \
  --functions-version 4 \
  --os-type Linux

# Deploy code
npm run build
func azure functionapp publish func-durable-loadtest

πŸ“ˆ Load Testing

See dedicated guide: LOAD_TESTING.md

Key considerations:

  • Use Premium plan (EP1+) to avoid cold starts skewing results
  • Disable authentication bypass (BYPASS_AUTH=false) for production testing
  • Enable Application Insights for detailed performance metrics
  • Test both session CRUD and orchestration patterns
  • Monitor storage throttling and RU consumption

βš™οΈ Scaling and host.json Tuning

See dedicated guide: docs/SCALING_AND_CONFIGURATION.md

The host.json file controls how the function app scales under load. Key settings:

Setting Default Purpose
maxConcurrentActivityFunctions 10 Activity parallelism per instance
maxConcurrentOrchestratorFunctions 10 Orchestrator parallelism per instance
partitionCount 4 Max instances for orchestration processing
extendedSessionsEnabled true Reduces replay overhead
maxConcurrentRequests 100 HTTP concurrency per instance
dynamicConcurrencyEnabled true Auto-tunes concurrency at runtime

The scaling guide covers:

  • What each host.json setting does and how to tune it
  • Scaling profiles for dev, load testing, and production
  • KQL queries to monitor scaling behavior in Application Insights
  • Common scaling pitfalls and how to fix them

πŸ—‚οΈ Project Structure

node-durable/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ activities/
β”‚   β”‚   └── activities.ts           # Activity functions for orchestrations
β”‚   β”œβ”€β”€ entities/
β”‚   β”‚   β”œβ”€β”€ index.ts                # Entity exports
β”‚   β”‚   β”œβ”€β”€ metricsEntity.ts        # Metrics tracking entity
β”‚   β”‚   └── sessionCounterEntity.ts # Session counter entity
β”‚   β”œβ”€β”€ functions/
β”‚   β”‚   β”œβ”€β”€ orchestrationApi.ts     # Orchestration HTTP endpoints
β”‚   β”‚   └── sessionApi.ts           # Session CRUD endpoints
β”‚   β”œβ”€β”€ models/
β”‚   β”‚   └── types.ts                # TypeScript interfaces
β”‚   β”œβ”€β”€ orchestrators/
β”‚   β”‚   β”œβ”€β”€ fanOutFanInOrchestrator.ts   # Parallel processing pattern
β”‚   β”‚   └── processWorkflowOrchestrator.ts  # Sequential workflow pattern
β”‚   β”œβ”€β”€ services/
β”‚   β”‚   β”œβ”€β”€ AzureTableStorageService.ts    # Table Storage implementation
β”‚   β”‚   β”œβ”€β”€ CosmosDbStorageService.ts      # Cosmos DB implementation
β”‚   β”‚   β”œβ”€β”€ IStorageService.ts             # Storage interface
β”‚   β”‚   └── StorageServiceFactory.ts       # Storage factory pattern
β”‚   β”œβ”€β”€ utils/
β”‚   β”‚   β”œβ”€β”€ appInsights.ts          # Application Insights integration
β”‚   β”‚   β”œβ”€β”€ authMiddleware.ts       # JWT authentication
β”‚   β”‚   └── durableClient.ts        # Durable Functions client helper
β”‚   └── index.ts                    # Application entry point
β”œβ”€β”€ infra/
β”‚   β”œβ”€β”€ main.tf                     # Terraform main configuration
β”‚   β”œβ”€β”€ variables.tf                # Terraform variables
β”‚   β”œβ”€β”€ outputs.tf                  # Terraform outputs
β”‚   β”œβ”€β”€ providers.tf                # Provider configuration
β”‚   β”œβ”€β”€ terraform.tfvars.example    # Example variables
β”‚   └── README.md                   # Infrastructure documentation
β”œβ”€β”€ docs/
β”‚   β”œβ”€β”€ AUTHENTICATION.md           # Authentication guide
β”‚   └── plan/                       # Project planning documents
β”œβ”€β”€ host.json                       # Azure Functions host configuration
β”œβ”€β”€ local.settings.json             # Local environment settings (not in git)
β”œβ”€β”€ local.settings.json.example     # Local settings template
β”œβ”€β”€ package.json                    # Node.js dependencies
β”œβ”€β”€ tsconfig.json                   # TypeScript configuration
└── README.md                       # This file

πŸ” Authentication

The application supports Azure Entra ID (Azure AD) authentication:

Development Mode

Set BYPASS_AUTH=true in local.settings.json to skip authentication during local testing.

Production Mode

  1. Create Azure AD App Registration
  2. Configure ENTRA_TENANT_ID, ENTRA_CLIENT_ID, ENTRA_AUDIENCE
  3. Set BYPASS_AUTH=false
  4. Include JWT token in Authorization: Bearer <token> header

See detailed guide: docs/AUTHENTICATION.md

πŸ› Troubleshooting

Azurite Won't Start

# Check if port 10000 is in use
netstat -ano | findstr :10000

# Kill process using port (Windows)
taskkill /PID <pid> /F

# Or specify different ports
azurite --blobPort 10001 --queuePort 10002 --tablePort 10003

Functions Won't Start

# Clean and rebuild
npm run clean
npm install
npm run build

# Check for compilation errors
npm run build -- --listFiles

"Table Not Found" Error

The application automatically creates the sessions table on first insert. If you see this error:

  1. Ensure Azurite is running
  2. Check AzureWebJobsStorage connection string is correct
  3. Verify storage service initialization logs

Storage Connection Issues

# Test Azurite connectivity
curl http://localhost:10000/

# Expected: Azure Blob Storage Emulator response

TypeScript Compilation Errors

# Check TypeScript version
npx tsc --version

# Reinstall dependencies
rm -rf node_modules package-lock.json
npm install

Orchestration Endpoints Not Working

This is a known issue. See Known Issues section above.

πŸ“ Development

Building the Project

npm run build          # Compile TypeScript
npm run watch          # Watch mode for development
npm run clean          # Remove dist/ directory

Code Quality

npm run lint           # Run ESLint (if configured)
npm test               # Run tests (if configured)

Adding New Endpoints

  1. Create function file in src/functions/
  2. Import in src/index.ts
  3. Rebuild: npm run build
  4. Test locally: func start

Adding Custom Telemetry

import { trackEvent, trackMetric } from './utils/appInsights';

// Track custom event
trackEvent('MyEvent', { userId: 'user123' }, { duration: 150 });

// Track custom metric
trackMetric('ItemsProcessed', 42, { operation: 'batch' });

πŸ“– Additional Resources

🀝 Contributing

  1. Create feature branch
  2. Make changes with tests
  3. Ensure npm run build succeeds
  4. Submit pull request

πŸ“„ License

MIT License - See LICENSE file for details

πŸ†˜ Support

For issues or questions:

  1. Check Troubleshooting section
  2. Review Known Issues
  3. Search existing issues in repository
  4. Create new issue with:
    • Error messages
    • Steps to reproduce
    • Environment details (OS, Node version, etc.)

Built with ❀️ using Azure Durable Functions

About

testing framework for durable functions configurations and scaling

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors