lex-arize

A LegionIO extension connecting Legion to Arize AX via REST v2 and GraphQL APIs. Covers 12 REST domains and 70 methods for managing AI integrations, annotations, datasets, evaluators, experiments, prompts, and organization resources. Includes AMQP actors that forward LLM audit and metering events to Arize.

Installation

gem 'lex-arize'

Configuration

{
  "arize": {
    "url": "https://app.arize.com",
    "api_key": "your-api-key",
    "space_id": "your-space-id",
    "project_id": "your-project-id",
    "forward_prompts": true,
    "forward_tools": true,
    "forward_metering": true,
    "dataset_id": null,
    "experiment_id": null
  }
}

Usage

client = Legion::Extensions::Arize::Client.new(
  url: 'https://app.arize.com',
  api_key: 'your-api-key'
)

# Datasets
client.list_datasets(space_id: 'sp1', limit: 10)
client.create_dataset(name: 'eval-set', description: 'Golden test set')
client.create_dataset_examples(dataset_id: 'ds1', examples: [{ input: { text: 'hi' }, output: { text: 'hello' } }])

# Evaluators
client.list_evaluators(space_id: 'sp1')
client.create_evaluator(name: 'hallucination-check')
client.list_evaluator_versions(evaluator_id: 'ev1')

# Experiments
client.create_experiment(dataset_id: 'ds1', name: 'baseline-run')
client.annotate_experiment_runs(experiment_id: 'exp1', annotations: [{ label: 'pass' }])

# Prompts
client.list_prompts(space_id: 'sp1')
client.create_prompt(name: 'system-prompt-v1')
client.create_prompt_version(prompt_id: 'pr1', template: 'You are helpful.')
client.resolve_prompt_label(prompt_id: 'pr1', label_name: 'production')

# Annotation Queues
client.list_annotation_queues(space_id: 'sp1')
client.annotate_record(annotation_queue_id: 'aq1', record_id: 'r1', label: 'correct', score: 0.95)

# Projects & Organizations
client.list_projects(space_id: 'sp1')
client.list_organizations

# Pagination — automatically fetches all cursor pages
all_datasets = client.paginate(:list_datasets, response_key: :datasets, space_id: 'sp1')

# GraphQL (for domains not yet in REST v2)
result = client.graphql(
  query: 'mutation($input: CreatePerformanceMonitorInput!) { createPerformanceMonitor(input: $input) { monitor { id } } }',
  variables: { input: { modelId: 'm1', name: 'latency-monitor' } }
)

# Error handling
begin
  client.get_dataset(dataset_id: 'nonexistent')
rescue Legion::Extensions::Arize::Errors::NotFoundError => e
  puts "Not found: #{e.message} (status: #{e.status})"
end

Supported Domains (12 REST v2)

Domain	Methods	Notes
AI Integrations	5	CRUDL
Annotation Configs	4	list, create, get, delete
Annotation Queues	10	CRUDL + records (list/create/delete/annotate/assign)
API Keys	4	list, create, delete, refresh
Datasets	8	CRUDL + examples (list/create/update/annotate)
Evaluators	8	CRUDL + versions (list/create/get)
Experiments	6	CRUDL + runs (list/annotate)
Organizations	5	CRUDL
Projects	4	list, create, get, delete
Prompts	11	CRUDL + versions + labels (resolve/set/remove)
Resource Restrictions	2	restrict, unrestrict
Role Bindings	3	create, get, update

AMQP Forwarding

When running as part of a LegionIO daemon with legion-transport, the extension creates Arize-specific queues bound to existing legion-llm audit and metering exchanges:

Queue	Exchange	Binding Key	Actor
arize.audit.prompts	llm.audit	audit.prompt.#	PromptForwarder
arize.audit.tools	llm.audit	audit.tool.#	ToolForwarder
arize.metering	llm.metering	metering.#	MeteringForwarder

Each actor transforms Legion event format to Arize API format and POSTs to the appropriate REST endpoint. Encrypted audit payloads (content_encoding: encrypted/cs) are automatically decrypted via Legion::Crypt before transformation.

Forwarding can be individually toggled via settings:

{
  "arize": {
    "forward_prompts": true,
    "forward_tools": true,
    "forward_metering": true
  }
}

Helpers

Pagination

all = client.paginate(:list_datasets, response_key: :datasets)

Follows next_cursor field automatically.

GraphQL

client.graphql(query: '{ projects { id name } }', variables: {})

Hook point for future GraphQL domains (Monitors, Dashboards, Metrics, Online Tasks).

Error Handling

rescue Legion::Extensions::Arize::Errors::AuthenticationError => e  # 401
rescue Legion::Extensions::Arize::Errors::AuthorizationError => e   # 403
rescue Legion::Extensions::Arize::Errors::NotFoundError => e        # 404
rescue Legion::Extensions::Arize::Errors::UnprocessableError => e   # 422
rescue Legion::Extensions::Arize::Errors::RateLimitError => e       # 429
rescue Legion::Extensions::Arize::Errors::ServerError => e          # 5xx
rescue Legion::Extensions::Arize::Errors::ArizeError => e           # all others

Development

bundle install
bundle exec rspec        # 168 examples, 0 failures
bundle exec rubocop      # 0 offenses

License

MIT