ARI Service with RTP Latency Measurement

This project implements precise RTP Round-Trip Latency measurements in an ARI service according to a specific architecture. It provides high-performance ARI service for accurate RTP latency measurement with SLA monitoring capabilities.

Table of Contents

• Architecture
• Components
• Environment Variables
• System Requirements
• Quick Start
  • Docker Setup
  • Local Setup
• Running Tests
  • Using run.sh
  • Using run_simple.sh
  • Using run_continuous.sh
  • Manual Execution
• Docker Usage
• Monitoring
• Latency Measurement
• Configuration
• Scripts
• Troubleshooting

Architecture

┌─────────────┐    RTP     ┌─────────────┐    RTP     ┌─────────────┐
│   Asterisk  │ ────────► │ ARI Service │ ────────► │ Echo Server │
│             │           │             │           │             │
│ - SIP calls │           │ - Parse RTP │           │ - Min delay │
│ - RTP gen   │           │ - Track seq │           │ - Pacing    │
└─────────────┘           │ - Measure   │           └─────────────┘
       ▲                  │   latency   │                    │
       │                  │ - SLA check │                    │
       │                  └─────────────┘                    │
       │                         ▲                           │
       │                         │                           │
       └─────────────────────────┼───────────────────────────┘
                            Round-trip
                            Measurement

Components

1. ARI Service (cmd/ari-service/main.go) - ARI client, StasisStart/End handling, externalMedia, bridge, channel manager
2. RTP Worker (internal/rtp/worker.go) - Concurrent UDP worker with channel-based processing, RTP parsing, send/receive, sequence number correlation, packet pacing
3. Metrics (internal/metrics/hist.go) - RTT histogram, p50/p95/p99, counters
4. Echo Server (cmd/echo/main.go) - Separate external process for echo loopback with TS pacing
5. Load Test (cmd/load_test_new/main.go) - Originate N calls, print results

Environment Variables

The project uses environment variables for configuration. These can be set in a .env file or exported directly.

# Asterisk ARI Configuration (for REST API calls from your service TO Asterisk)
ASTERISK_HOST=localhost
ASTERISK_PORT=8088
ASTERISK_USERNAME=ari
ASTERISK_PASSWORD=ari
ASTERISK_APP_NAME=ari-service

# ARI Configuration (for WebSocket authentication - Asterisk connecting TO your service)
ARI_URL=localhost:8088
ARI_USER=asterisk
ARI_PASS=asterisk
APP_NAME=ari-app

# RTP Configuration
BIND_IP=0.0.0.0
PORT_RANGE=21000-31000

# Echo Server Configuration
ECHO_HOST=localhost
ECHO_PORT=4000

# Metrics
METRICS_INTERVAL_SEC=5

# Load Test Configuration
LOAD_TEST_CONCURRENT_CALLS=10
LOAD_TEST_DURATION_SECONDS=60
LOAD_TEST_CALL_DURATION_SECONDS=30
LOAD_TEST_ENDPOINT=Local/echo@ari-context
LOAD_TEST_REPORT=reports/load_test_report.json

# Service Configuration
SERVICE_BIND_IP=0.0.0.0
SERVICE_PORT=9090

System Requirements

• Go 1.16+
• Docker and Docker Compose (for Docker setup)
• Asterisk with ARI enabled (for local setup)
• Proper UDP buffer tuning (sysctl settings recommended)

Quick Start

Docker Setup

The easiest way to run this project is using Docker:

# Build and start the Docker container
docker-compose up -d

# Wait for services to initialize (about 10 seconds)
sleep 10

# Check if services are running
docker-compose exec asterisk curl -s http://localhost:9090/health

# Run the load test
docker-compose exec asterisk bash -c "cd /app && ./run.sh"

# View metrics
docker-compose exec asterisk curl -s http://localhost:9090/metrics | jq '.'

# Stop services
docker-compose down

Local Setup

To run locally without Docker:

# Export environment variables
export ASTERISK_HOST=localhost
export ASTERISK_PORT=8088
export ASTERISK_USERNAME=ari
export ASTERISK_PASSWORD=ari
export ARI_URL=localhost:8088
export ARI_USER=asterisk
export ARI_PASS=asterisk
export APP_NAME=ari-app
export BIND_IP=0.0.0.0
export PORT_RANGE=21000-31000
export ECHO_HOST=localhost
export ECHO_PORT=4000
export METRICS_INTERVAL_SEC=5

# Build all components
go build -o bin/ari-service ./cmd/ari-service
go build -o bin/echo-server ./cmd/echo
go build -o bin/load-test-new ./cmd/load_test_new

# Start ARI service
./bin/ari-service &

# Start echo server as separate process
./bin/echo-server &

# Run load test
./bin/load-test-new --count=10 --duration-ms=60000

Running Tests

Using run.sh

The run.sh script provides a complete workflow for running tests:

# Run regular load test with default configuration (10 calls, 60s duration, 30s call duration)
./run.sh

# Run production test (50 calls, 300s duration, 60s call duration)
./run.sh prod

# Run 100 calls test (100 calls, 600s duration, 1800s call duration)
./run.sh 100call

Using run_simple.sh

The run_simple.sh script allows for custom parameters:

# Run with custom parameters
./run_simple.sh --count=30 --duration-ms=30000 --delay-between-ms=100

# Run with different configuration
./run_simple.sh --count=100 --duration-ms=60000 --delay-between-ms=50

Using run_continuous.sh

The run_continuous.sh script maintains a constant number of concurrent calls:

# Run continuous load test with default configuration (10 concurrent calls, 30s call duration)
./run_continuous.sh

# Run with custom parameters
./run_continuous.sh --concurrent-calls=50 --call-duration-seconds=60 --delay-between-calls-ms=200

# Run with different configuration
./run_continuous.sh --concurrent-calls=100 --call-duration-seconds=120 --delay-between-calls-ms=100

The continuous load test will:

• Maintain the specified number of concurrent calls
• Automatically start new calls when existing ones complete
• Run indefinitely until stopped with Ctrl+C
• Report metrics at regular intervals

Manual Execution

You can also run components manually:

# Start services
./bin/ari-service &
./bin/echo-server &

# Run load test with specific parameters
./bin/load-test-new \
  --count=10 \
  --duration-ms=60000 \
  --delay-between-ms=100

Docker Usage

Building Docker Images

# Build the Docker image
docker-compose build

Running with Docker Compose

# Start services in background
docker-compose up -d

# View logs
docker-compose logs -f

# Execute commands in the container
docker-compose exec asterisk bash

# Stop services
docker-compose down

Running Tests in Docker

# Run the default test configuration
docker-compose exec asterisk bash -c "cd /app && ./run.sh"

# Run with custom configuration by updating .env file first
docker-compose exec asterisk bash -c "cd /app && ./run_simple.sh --count=50 --duration-ms=120000"

Environment Variables in Docker

Environment variables can be set in multiple ways when using Docker:

1. In docker-compose.yml: Define environment variables directly
2. In .env file: Variables will be automatically loaded
3. At runtime: Pass variables when running docker-compose

Example of setting variables at runtime:

LOAD_TEST_CONCURRENT_CALLS=50 LOAD_TEST_DURATION_SECONDS=300 docker-compose up -d

Monitoring

The ARI service exposes metrics via HTTP endpoints:

• Health check: curl http://localhost:9090/health
• Metrics: curl http://localhost:9090/metrics

Example metrics output:

{
  "total_channels": 12,
  "active_channels": 12,
  "total_latencies": 33682,
  "p50_latency": 11,
  "p95_latency": 17,
  "p99_latency": 19,
  "max_latency": 106,
  "avg_latency": 11.414820972626329,
  "late_ratio": 0.9991392359977442,
  "packet_loss_ratio": 0,
  "timestamp": "2025-09-01T09:30:15.979076055Z"
}

Latency Measurement

The system measures RTP round-trip latency using the following approach:

1. StasisStart: Answer → externalMedia (both, ulaw, udp, rtp, external_host=BIND_IP:PORT) → bridge(mixing) + add(client, externalMedia)

2. One UDP worker per channel (concurrent model):

   • From Asterisk: Parse RTP (12 bytes), paced send → echo, RecordSend(seq)
   • From echo: Parse, GetLatency(seq), RTT calculation, send → Asterisk (from the same local port)

3. MVP Metrics: p50/p95/p99/max RTT, drops by Seq (echo→you)

4. Teardown on StasisEnd: Stop worker, close socket, return port to pool, clean state

Configuration

Environment Variables Reference

Variable	Description	Default Value
ASTERISK_HOST	Asterisk host for REST API calls	localhost
ASTERISK_PORT	Asterisk port for REST API calls	8088
ASTERISK_USERNAME	Username for REST API calls	ari
ASTERISK_PASSWORD	Password for REST API calls	ari
ARI_URL	ARI WebSocket URL	localhost:8088
ARI_USER	Username for WebSocket auth	asterisk
ARI_PASS	Password for WebSocket auth	asterisk
APP_NAME	ARI application name	ari-app
BIND_IP	IP to bind RTP ports	0.0.0.0
PORT_RANGE	Range of ports for RTP	21000-31000
ECHO_HOST	Echo server host	localhost
ECHO_PORT	Echo server port	4000
METRICS_INTERVAL_SEC	Metrics reporting interval	5
LOAD_TEST_CONCURRENT_CALLS	Number of concurrent calls	10
LOAD_TEST_DURATION_SECONDS	Test duration in seconds	60
LOAD_TEST_CALL_DURATION_SECONDS	Call duration in seconds	30
LOAD_TEST_ENDPOINT	Endpoint to call	Local/echo@ari-context
SERVICE_PORT	Port for internal service	9090

Sysctl Optimization

For production use, increase UDP buffer sizes:

# Add to /etc/sysctl.conf
net.core.rmem_max = 268435456  # 256MB
net.core.wmem_max = 268435456  # 256MB
net.core.rmem_default = 262144 # 256KB
net.core.wmem_default = 262144 # 256KB
net.ipv4.udp_rmem_min = 262144 # 256KB
net.ipv4.udp_wmem_min = 262144 # 256KB

# Apply settings
sudo sysctl -p

Ulimit Settings

For high-performance operation, increase file descriptor limits:

# Add to /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536

# Or run with:
ulimit -n 65536

Scripts

The project includes several utility scripts:

Main Scripts

• run.sh - Complete workflow script with different test configurations
• run_simple.sh - Simple script with customizable parameters
• run_continuous.sh - Continuous load test that maintains constant concurrent calls
• test.sh - Basic test script
• cleanup.sh - Cleanup script for logs and reports

Test Scripts

• run_comprehensive_test.sh - Comprehensive testing script
• run_extended_test.sh - Extended testing with multiple scenarios
• test_env_configurations.sh - Test different environment configurations
• test_sla.sh - SLA validation testing

Monitoring Scripts

• demo_monitoring.sh - Demonstration of monitoring capabilities
• monitor_resources.sh - Resource monitoring script
• analyze_test_results.sh - Test result analysis

Docker Scripts

• run_docker_env.sh - Run tests with Docker environment variables
• run_docker_test.sh - Docker testing script
• run_docker_tests.sh - Multiple Docker tests

Troubleshooting

Common Issues

1. Services not starting: Check Docker logs with docker-compose logs
2. Connection refused: Ensure ports are not blocked by firewall
3. High latency: Check network connectivity and system resources
4. Packet loss: Increase UDP buffer sizes and check network infrastructure

Debugging

# Check service health
curl http://localhost:9090/health

# View detailed logs
tail -f logs/ari-service.log
tail -f logs/echo-server.log

# Check system resources
docker-compose exec asterisk top

Performance Tuning

1. Increase UDP buffers: Apply sysctl optimizations
2. Adjust file limits: Increase ulimit for file descriptors
3. CPU affinity: Pin processes to specific CPU cores
4. Network tuning: Optimize network interface settings

License

MIT License - see LICENSE file for details.# ari-echo