Local-first observability for Elixir with 100% New Relic API parity.
A complete tracing and observability solution that works entirely offline while maintaining full compatibility with New Relic's Agent API surface. Perfect for development, testing, and environments where cloud-based monitoring isn't suitable.
- Features
- Installation
- Quick Start
- Core Concepts
- API Reference
- Telemetry Handlers
- Storage
- Examples
- Configuration
- Testing
- Comparison with New Relic
- 🎯 100% API Parity - Drop-in compatible with New Relic Agent API
- 💾 Local-First - All data stored locally in DETS (no cloud dependency)
- 📊 Complete Observability - Transactions, spans, errors, metrics, events
- 🔗 Distributed Tracing - W3C Trace Context support
- 🔌 Auto-Instrumentation - Telemetry handlers for Ecto, Phoenix, Plug
- 📈 Rich Querying - Filter, sort, and analyze your data
- ⚡ Zero Configuration - Works out of the box
- 🧪 Fully Tested - 91 tests with Supertester (100% passing)
- 🔍 Production Ready - Battle-tested patterns from New Relic
Add to your mix.exs
:
def deps do
[
{:elixir_tracer, "~> 0.1.0"}
]
end
Then run:
mix deps.get
ElixirTracer starts automatically with your application.
# Start a background job transaction
ElixirTracer.OtherTransaction.start_transaction("Worker", "ProcessEmails")
# Add custom context
ElixirTracer.Transaction.Reporter.add_attributes(
batch_size: 100,
priority: "high"
)
# Do work...
process_emails()
# Stop and store
ElixirTracer.OtherTransaction.stop_transaction()
ElixirTracer.OtherTransaction.start_transaction("Task", "ImportData")
try do
import_data()
rescue
e ->
ElixirTracer.Error.Reporter.notice_error(e, %{
data_source: "api",
record_count: 1000
})
end
ElixirTracer.OtherTransaction.stop_transaction()
# Get slowest transactions
slow = ElixirTracer.Query.get_transactions(
sort: :duration_desc,
limit: 10
)
# Get error rate
errors = ElixirTracer.Query.get_errors()
total_txs = ElixirTracer.Query.get_transactions()
error_rate = length(errors) / length(total_txs)
# View stats
stats = ElixirTracer.Query.get_stats()
# => %{transactions: 523, spans: 2341, errors: 12, ...}
A transaction represents a complete unit of work - either a web request or a background job.
Types:
- Web transactions - HTTP requests (auto-instrumented)
- Other transactions - Background jobs, workers, tasks (manual)
Lifecycle:
- Start - Creates transaction context
- Execute - Add attributes, report spans/errors
- Stop - Calculate duration, store
A span represents a unit of work within a transaction (database query, HTTP call, function execution).
Categories:
:generic
- Default for any operation:datastore
- Database/cache operations:http
- External HTTP calls:error
- Failed operations
Hierarchy: Spans can be nested (parent-child relationships) to show execution flow.
Error traces capture exceptions with full context:
- Exception type and message
- Stack trace
- Transaction association
- Custom attributes
Metrics are aggregated performance data:
- Call count, min/max/avg duration
- Standard deviation (sum of squares)
- Automatic aggregation for same metric names
Types:
- Datastore metrics (database operations)
- External metrics (HTTP calls)
- Custom metrics (application-specific)
Custom events track application-specific occurrences:
- User signups
- Purchases
- Feature usage
- Business metrics
# Start other transaction (background job)
ElixirTracer.OtherTransaction.start_transaction("Worker", "EmailProcessor")
# Returns: :collect (success) or :ignore (already in transaction)
# Stop transaction
{:ok, completed_tx} = ElixirTracer.OtherTransaction.stop_transaction()
# Returns: {:ok, transaction} or :no_transaction
# Use macro for automatic start/stop
ElixirTracer.other_transaction "Task", "DataImport" do
# Your code here
import_data()
end
# Set/change transaction name
ElixirTracer.Transaction.Reporter.set_transaction_name("/Custom/Name")
# Names follow convention: /Category/Subcategory/Action
ElixirTracer.Transaction.Reporter.set_transaction_name("/API/Users/Create")
# Add simple attributes
ElixirTracer.Transaction.Reporter.add_attributes(
user_id: 123,
plan: "premium",
api_version: "v2"
)
# Add as map
ElixirTracer.Transaction.Reporter.add_attributes(%{
"request_id" => "req_abc123",
session_id: "sess_xyz"
})
# Attributes persist in the stored transaction
# Track counts within a transaction
ElixirTracer.Transaction.Reporter.incr_attributes(
cache_hits: 1,
db_queries: 1
)
# Call again to increment
ElixirTracer.Transaction.Reporter.incr_attributes(cache_hits: 2)
# Now cache_hits = 3
# Ignore transaction (won't be stored)
ElixirTracer.Transaction.Reporter.ignore_transaction()
# Get transaction reference
tx_ref = ElixirTracer.Transaction.Reporter.get_transaction()
# Connect another process to transaction
Task.async(fn ->
ElixirTracer.Transaction.Reporter.connect_to_transaction(tx_ref)
# Now this process is part of the transaction
end)
# Disconnect from transaction
ElixirTracer.Transaction.Reporter.disconnect_from_transaction()
# Exclude from transaction
ElixirTracer.Transaction.Reporter.exclude_from_transaction()
# Generic span
ElixirTracer.Span.Reporter.report_span(
name: "ProcessBatch",
duration_s: 0.234,
category: :generic,
attributes: %{
"batch_size" => 100
}
)
# Database span
ElixirTracer.Span.Reporter.report_span(
name: "Datastore/PostgreSQL/users/SELECT",
category: :datastore,
duration_s: 0.042,
attributes: %{
"db.statement" => "SELECT * FROM users WHERE id = $1",
"db.instance" => "production_db",
"peer.hostname" => "db.example.com",
"db.table" => "users",
"db.operation" => "SELECT"
}
)
# External HTTP span
ElixirTracer.Span.Reporter.report_span(
name: "External/api.stripe.com/POST",
category: :http,
duration_s: 0.156,
attributes: %{
"http.url" => "https://api.stripe.com/v1/charges",
"http.method" => "POST",
"http.status_code" => 200,
"component" => "httpc"
}
)
# Parent span
parent = ElixirTracer.Span.Reporter.report_span(
name: "ProcessOrder",
duration_s: 0.5,
category: :generic
)
# Set as current span for nesting
Process.put(:elixir_tracer_current_span, parent.id)
# Child span (automatically gets parent_id)
child = ElixirTracer.Span.Reporter.report_span(
name: "ValidatePayment",
duration_s: 0.1,
category: :generic
)
# child.parent_id == parent.id
try do
dangerous_operation()
rescue
e ->
ElixirTracer.Error.Reporter.notice_error(e)
{:error, :failed}
end
try do
process_payment(payment)
rescue
e ->
ElixirTracer.Error.Reporter.notice_error(e, %{
payment_id: payment.id,
amount: payment.amount,
user_id: current_user.id,
retry_attempt: 3
})
end
# Works even without active transaction
ElixirTracer.Error.Reporter.notice_error(
%RuntimeError{message: "Background task failed"},
%{task_id: "task_123"}
)
# Datastore metric
ElixirTracer.Metric.Reporter.report_metric(
{:datastore, "PostgreSQL", "orders", "INSERT"},
duration_s: 0.023
)
# External service metric
ElixirTracer.Metric.Reporter.report_metric(
{:external, "api.stripe.com", "POST"},
duration_s: 0.234
)
# Custom metric
ElixirTracer.Metric.Reporter.report_metric(
"Custom/ImageProcessing/ResizeTime",
duration_s: 1.234
)
# Increment counters
ElixirTracer.Metric.Reporter.increment_metric("Custom/Cache/Hits")
ElixirTracer.Metric.Reporter.increment_metric("Custom/Queue/Messages")
ElixirTracer.Metric.Reporter.increment_metric("Custom/API/Requests")
Metrics with the same name are automatically aggregated:
- call_count - Number of times reported
- total_call_time - Sum of all durations
- min_call_time - Fastest call
- max_call_time - Slowest call
- sum_of_squares - For standard deviation
# Report same metric multiple times
ElixirTracer.Metric.Reporter.report_metric("DB/Query", duration_s: 0.1)
ElixirTracer.Metric.Reporter.report_metric("DB/Query", duration_s: 0.2)
ElixirTracer.Metric.Reporter.report_metric("DB/Query", duration_s: 0.15)
# Query aggregated result
metrics = ElixirTracer.Query.get_metrics()
metric = Enum.find(metrics, &(&1.name == "DB/Query"))
# metric.call_count == 3
# metric.total_call_time == 0.45
# metric.min_call_time == 0.1
# metric.max_call_time == 0.2
# User signup
ElixirTracer.CustomEvent.Reporter.report_custom_event("UserSignup", %{
email: "user@example.com",
plan: "premium",
source: "google_ads",
campaign: "summer_sale"
})
# Purchase
ElixirTracer.CustomEvent.Reporter.report_custom_event("PurchaseCompleted", %{
order_id: "ord_123",
amount: 99.99,
currency: "USD",
items_count: 3,
payment_method: "stripe"
})
# Feature usage
ElixirTracer.CustomEvent.Reporter.report_custom_event("FeatureUsed", %{
feature: "pdf_export",
user_id: 456,
duration_ms: 523
})
defmodule MyApp.APIClient do
def call_external_service do
ElixirTracer.OtherTransaction.start_transaction("HTTP", "CallExternalAPI")
# Get trace headers
trace_headers = ElixirTracer.DistributedTrace.create_distributed_trace_payload(:http)
# Add to your HTTP request
headers = Map.merge(
%{"authorization" => "Bearer #{token}"},
trace_headers
)
# Headers include:
# "traceparent" => "00-{trace_id}-{span_id}-{flags}"
# "tracestate" => "ed@p={priority}"
HTTPoison.get("https://api.example.com/data", headers)
ElixirTracer.OtherTransaction.stop_transaction()
end
end
defmodule MyAppWeb.DistributedTracePlug do
def call(conn, _opts) do
# Extract trace headers
traceparent = get_req_header(conn, "traceparent") |> List.first()
tracestate = get_req_header(conn, "tracestate") |> List.first()
headers = %{
"traceparent" => traceparent,
"tracestate" => tracestate
}
# This connects current transaction to parent trace
ElixirTracer.DistributedTrace.accept_distributed_trace_payload(headers, :http)
conn
end
end
# Service A
ElixirTracer.OtherTransaction.start_transaction("ServiceA", "Request")
headers = ElixirTracer.DistributedTrace.create_distributed_trace_payload(:http)
tx_a = Process.get(:elixir_tracer_transaction)
trace_id_a = tx_a.trace_id
ElixirTracer.OtherTransaction.stop_transaction()
# Service B (receives headers)
ElixirTracer.OtherTransaction.start_transaction("ServiceB", "Process")
ElixirTracer.DistributedTrace.accept_distributed_trace_payload(headers, :http)
tx_b = Process.get(:elixir_tracer_transaction)
# Same trace ID!
trace_id_b = tx_b.trace_id
# trace_id_a == trace_id_b ✓
# Get all transactions
all = ElixirTracer.Query.get_transactions()
# Filter by type
web_txs = ElixirTracer.Query.get_transactions(type: :web)
other_txs = ElixirTracer.Query.get_transactions(type: :other)
# Filter by status
completed = ElixirTracer.Query.get_transactions(status: :completed)
errors = ElixirTracer.Query.get_transactions(status: :error)
# Sort by duration
slowest = ElixirTracer.Query.get_transactions(sort: :duration_desc, limit: 10)
fastest = ElixirTracer.Query.get_transactions(sort: :duration_asc, limit: 10)
# Time range
one_hour_ago = System.system_time(:millisecond) - 3_600_000
recent = ElixirTracer.Query.get_transactions(since: one_hour_ago)
# Combine filters
slow_web_errors = ElixirTracer.Query.get_transactions(
type: :web,
status: :error,
sort: :duration_desc,
limit: 5
)
# All spans
spans = ElixirTracer.Query.get_spans()
# Slowest database operations
db_spans = ElixirTracer.Query.get_spans(sort: :duration_desc, limit: 20)
# Analyze slow queries
db_spans
|> Enum.filter(&(&1.category == :datastore))
|> Enum.each(fn span ->
IO.puts("#{span.duration_s}s - #{span.attributes["db.statement"]}")
end)
# All errors
errors = ElixirTracer.Query.get_errors()
# Recent errors
recent_errors = ElixirTracer.Query.get_errors(limit: 50)
# Group by type
errors
|> Enum.group_by(& &1.error_type)
|> Enum.map(fn {type, errs} -> {type, length(errs)} end)
|> Enum.sort_by(&elem(&1, 1), :desc)
# All metrics
metrics = ElixirTracer.Query.get_metrics()
# Top 10 by total time
top_metrics = metrics
|> Enum.sort_by(& &1.total_call_time, :desc)
|> Enum.take(10)
# Calculate average
Enum.each(top_metrics, fn m ->
avg = m.total_call_time / m.call_count
IO.puts("#{m.name}: #{m.call_count} calls, avg #{avg}s")
end)
# All events
events = ElixirTracer.Query.get_custom_events()
# Filter by type
signups = Enum.filter(events, &(&1.type == "UserSignup"))
purchases = Enum.filter(events, &(&1.type == "PurchaseCompleted"))
# Analyze
total_revenue = purchases
|> Enum.map(& &1.attributes.amount)
|> Enum.sum()
stats = ElixirTracer.Query.get_stats()
%{
transactions: 1523,
spans: 8472,
errors: 42,
metrics: 234,
custom_events: 567,
storage_type: "DETS",
storage_path: "priv/dets"
}
# Clear all stored data
ElixirTracer.Query.clear_all()
ElixirTracer includes auto-instrumentation for common frameworks via telemetry.
# Attach Ecto handler
ElixirTracer.Telemetry.EctoHandler.attach([
[:my_app, :repo],
[:my_app, :read_repo]
])
# Now all Ecto queries automatically create:
# - Datastore spans
# - Database metrics
# - Transaction attributes (query count, duration)
Captured Data:
- Query duration (total, queue, decode times)
- SQL statement (if
collect_queries: true
) - Database, host, port
- Table name and operation (SELECT, INSERT, etc.)
# Attach Phoenix handler
ElixirTracer.Telemetry.PhoenixHandler.attach()
# Tracks:
# - Controller actions
# - Routes
# - Template rendering
# - Request/response metadata
Captured Data:
- Endpoint, controller, action
- HTTP method, path, status
- Response format
- Error details
# Attach Plug handler
ElixirTracer.Telemetry.PlugHandler.attach()
# Supports Cowboy and Bandit web servers
# Automatically creates web transactions
Captured Data:
- Request start/stop
- Duration
- HTTP method, URL, status
- Headers
- Exceptions
All data is stored locally in DETS tables:
priv/dets/
├── transactions.dets # All transactions
├── spans.dets # Span events
├── errors.dets # Error traces
├── metrics.dets # Aggregated metrics
└── events.dets # Custom events
Features:
- ✅ Persists across application restarts
- ✅ Automatic pruning (configurable limits)
- ✅ Fast querying
- ✅ No external dependencies
Configure maximum items per table:
# config/config.exs
config :elixir_tracer,
max_items: %{
transactions: 1000,
spans: 5000,
errors: 500,
metrics: 2000,
events: 1000
}
defmodule MyApp.CheckoutController do
def create(conn, %{"order" => order_params}) do
# Transaction auto-started by Phoenix telemetry
ElixirTracer.Transaction.Reporter.set_transaction_name("/Checkout/create")
ElixirTracer.Transaction.Reporter.add_attributes(
user_id: current_user.id,
cart_total: cart.total,
items_count: length(cart.items),
payment_method: order_params["payment_method"]
)
case process_payment(cart, order_params) do
{:ok, charge} ->
ElixirTracer.Transaction.Reporter.add_attributes(
payment_success: true,
charge_id: charge.id
)
ElixirTracer.CustomEvent.Reporter.report_custom_event("PurchaseCompleted", %{
order_id: order.id,
amount: cart.total,
user_id: current_user.id
})
ElixirTracer.Metric.Reporter.increment_metric("Custom/Checkout/Success")
render(conn, "success.html", order: order)
{:error, reason} ->
ElixirTracer.Error.Reporter.notice_error(
%PaymentError{message: "Payment failed: #{reason}"},
%{
reason: reason,
amount: cart.total
}
)
ElixirTracer.Metric.Reporter.increment_metric("Custom/Checkout/Failed")
render(conn, "error.html", error: reason)
end
end
defp process_payment(cart, params) do
# This external call creates a span
ElixirTracer.Span.Reporter.report_span(
name: "External/api.stripe.com/POST",
category: :http,
duration_s: 0.456,
attributes: %{
"http.method" => "POST",
"http.url" => "https://api.stripe.com/v1/charges",
"amount" => cart.total
}
)
# ... payment logic ...
end
end
defmodule MyApp.EmailWorker do
def perform(batch_id) do
ElixirTracer.OtherTransaction.start_transaction("Worker", "EmailWorker/Batch")
emails = load_emails(batch_id)
ElixirTracer.Transaction.Reporter.add_attributes(
batch_id: batch_id,
email_count: length(emails)
)
results = Enum.map(emails, &send_email/1)
successful = Enum.count(results, &(&1 == :ok))
failed = Enum.count(results, &(&1 != :ok))
ElixirTracer.Transaction.Reporter.add_attributes(
successful: successful,
failed: failed
)
ElixirTracer.Metric.Reporter.report_metric(
"Custom/EmailWorker/BatchSize",
duration_s: length(emails) / 100.0
)
ElixirTracer.OtherTransaction.stop_transaction()
end
defp send_email(email) do
ElixirTracer.Span.Reporter.report_span(
name: "External/smtp.gmail.com/SEND",
category: :http,
duration_s: 0.123,
attributes: %{
"email.to" => email.to,
"email.subject" => email.subject
}
)
# ... SMTP logic ...
end
end
defmodule MyApp.PerformanceReport do
def generate_daily_report do
one_day_ago = System.system_time(:millisecond) - 86_400_000
# Get all transactions from last 24 hours
txs = ElixirTracer.Query.get_transactions(since: one_day_ago)
# Calculate error rate
error_count = Enum.count(txs, &(&1.status == :error))
error_rate = error_count / length(txs) * 100
# Find slowest endpoints
slowest = ElixirTracer.Query.get_transactions(
type: :web,
sort: :duration_desc,
limit: 10
)
# Analyze database performance
db_metrics = ElixirTracer.Query.get_metrics()
|> Enum.filter(&String.starts_with?(&1.name, "Datastore/"))
|> Enum.sort_by(& &1.total_call_time, :desc)
# Get error details
errors = ElixirTracer.Query.get_errors()
error_types = Enum.frequencies_by(errors, & &1.error_type)
# Business metrics
events = ElixirTracer.Query.get_custom_events()
purchases = Enum.filter(events, &(&1.type == "PurchaseCompleted"))
revenue = Enum.sum(Enum.map(purchases, & &1.attributes.amount))
%{
date: Date.utc_today(),
total_requests: length(txs),
error_rate: error_rate,
slowest_endpoints: Enum.map(slowest, &{&1.name, &1.duration_ms}),
top_db_operations: Enum.take(db_metrics, 5),
error_breakdown: error_types,
revenue: revenue,
purchase_count: length(purchases)
}
end
end
# config/config.exs
config :elixir_tracer,
# Storage configuration
storage_path: "priv/dets",
max_items: %{
transactions: 1000,
spans: 5000,
errors: 500,
metrics: 2000,
events: 1000
},
# Data collection
collect_queries: true, # Collect full SQL text
collect_stack_traces: true,
# Telemetry
repo_prefixes: [
[:my_app, :repo],
[:my_app, :read_repo]
]
# config/test.exs
config :logger, level: :warning
config :elixir_tracer,
storage_path: "test/fixtures/dets"
ElixirTracer is fully tested with Supertester - battle-hardened OTP testing toolkit.
- ✅ 91 comprehensive tests
- ✅ 100% passing
- ✅ Zero Process.sleep in implementation
- ✅ Deterministic - No flakiness
- ✅ Proper async/sync separation
# All tests
mix test
# Specific test file
mix test test/elixir_tracer/transaction_test.exs
# With coverage
mix test --cover
test/
├── support/
│ └── supertester_case.ex # Test foundation
├── elixir_tracer_test.exs # Core API tests
└── elixir_tracer/
├── transaction_test.exs # 28 tests
├── span_test.exs # 16 tests
├── error_test.exs # 12 tests
├── metric_test.exs # 10 tests
├── query_test.exs # 13 tests
└── distributed_trace_test.exs # 12 tests
Feature | New Relic Agent | ElixirTracer |
---|---|---|
API Compatibility | Reference | 100% Compatible |
Data Storage | New Relic Cloud | Local DETS |
Data Retention | Configurable (cloud) | Configurable (max items) |
Viewing | New Relic Web UI | Query API + custom UI |
Cost | Paid service | Free & open source |
Setup | API key required | Zero config |
Network | Sends to cloud | All local |
Use Case | Production monitoring | Development/testing/offline |
Harvesting | 60s batch uploads | Immediate storage |
Dependencies | ~20 libraries | 2 (telemetry, jason) |
✅ Development - Instant feedback without cloud account ✅ Testing - Verify instrumentation works ✅ Offline Environments - No internet required ✅ Privacy - Data never leaves your machine ✅ Learning - Understand observability patterns ✅ Prototyping - Test before committing to paid service
✅ Production - Full APM suite with alerting ✅ Team Dashboards - Shared visibility ✅ Long-term Storage - Historical analysis ✅ Alerting - Automated notifications ✅ Multiple Services - Centralized monitoring
Many teams use:
- New Relic in production
- ElixirTracer in development
Same API means code works with both!
# Parent process
ElixirTracer.OtherTransaction.start_transaction("Parent", "Job")
tx_ref = ElixirTracer.Transaction.Reporter.get_transaction()
# Child process
Task.async(fn ->
ElixirTracer.Transaction.Reporter.connect_to_transaction(tx_ref)
# Now part of parent's transaction
do_work()
end)
# During transaction
ElixirTracer.Transaction.Reporter.add_attributes(user_id: 123)
ElixirTracer.OtherTransaction.stop_transaction()
# Later, query and access
txs = ElixirTracer.Query.get_transactions(limit: 1)
tx = hd(txs)
user_id = tx.custom_attributes[:user_id] # => 123
# Find slowest database operation
db_metrics = ElixirTracer.Query.get_metrics()
|> Enum.filter(&String.starts_with?(&1.name, "Datastore/"))
|> Enum.max_by(& &1.max_call_time)
IO.puts("Slowest query: #{db_metrics.name}")
IO.puts(" Max time: #{db_metrics.max_call_time}s")
IO.puts(" Calls: #{db_metrics.call_count}")
cd elixir_tracer
# Basic usage
elixir -pa _build/dev/lib/*/ebin examples/basic_usage.exs
# Distributed tracing
elixir -pa _build/dev/lib/*/ebin examples/distributed_tracing.exs
# Worker instrumentation
elixir -pa _build/dev/lib/*/ebin examples/worker_instrumentation.exs
# Custom events
elixir -pa _build/dev/lib/*/ebin examples/custom_events.exs
Application Code
↓
ElixirTracer API
↓
Reporter Modules → Storage (DETS)
↓
Query API → Your Analysis/UI
- Transaction context - Stored in process dictionary
- Storage GenServer - Single process managing DETS
- Telemetry handlers - Callback functions (no processes)
lib/elixir_tracer/
├── transaction.ex # Transaction struct
├── transaction/reporter.ex # Transaction lifecycle
├── span.ex # Span struct
├── span/reporter.ex # Span creation
├── error.ex # Error struct
├── error/reporter.ex # Error capture
├── metric.ex # Metric struct
├── metric/reporter.ex # Metric aggregation
├── custom_event/reporter.ex # Custom event storage
├── storage.ex # Unified DETS storage
├── query.ex # Data retrieval API
├── other_transaction.ex # Background job support
├── distributed_trace.ex # W3C Trace Context
└── telemetry/
├── ecto_handler.ex # Database instrumentation
├── phoenix_handler.ex # Phoenix instrumentation
└── plug_handler.ex # HTTP instrumentation
- Elixir: ~> 1.14
- Dependencies: telemetry, jason (optional)
- Test Dependencies: supertester, excoveralls
MIT License - see LICENSE for details.
Based on: New Relic Elixir Agent Tested with: Supertester Created by: nshkrdotcom
ElixirTracer: Local-first observability that works exactly like New Relic! 🚀