Artificial Analysis: Detailed Introduction

[ikaijua]

Artificial Analysis: Detailed Introduction

Artificial Analysis (AA) compares AI model capability, task cost, and speed to help you choose models and inference providers.

• Coverage: language models, coding agents, images, video, and speech.
• Audience: developers, teams choosing models, and readers interpreting benchmark claims.
• Access: free public leaderboards, a free data API, and commercial data services.
• Workflow: shortlist for your task, compare capability and efficiency, then validate on your own examples.

Official account & key contributors on X

• Artificial Analysis (@ArtificialAnlys)
• Micah Hill-Smith — cofounder and CEO (@_micah_h)
• George Cameron — cofounder and CPO (@grmcameron)

Related platform: Arena: Detailed Introduction

Table of Contents

1. What it is and who it helps
2. Where to start
3. Reading the Intelligence Index
4. Understanding cost and speed
5. A practical model-selection workflow
6. Specialized leaderboards
7. Arena comparison and common pitfalls
8. Access, data API, and attribution
9. Official references

Use the contents to jump to each section. If an in-page link does not respond, open it in a new tab.

[ikaijua]

What it is and who it helps

Artificial Analysis (AA) compares AI models and inference providers to help you choose a model for a particular job.

• Choose a model by considering capability, cost, and speed together.
• Choose a provider: the same model can be available through different API endpoints, with different performance.
• Explore specialized evaluations for coding agents, images, video, and speech.

It is useful for API developers, teams making procurement decisions, and readers interpreting model-release claims.

Start with the model comparison page, shortlist candidates, and inspect relevant evaluations and costs. Public leaderboards are browsable; data access and commercial services are covered below.

AA is a benchmarking company. Its scores help narrow the field; they do not guarantee that the highest-scoring model will perform best on your own workflow.

[ikaijua]

Where to start

Your goal	Official page
Compare language-model capability, price, and speed	Models
Compare coding agents	Coding Agents
Explore image generation and editing	Image Arena
Compare video models	Video Arena
Understand scoring	Methodology
Retrieve data programmatically	API Reference

On your first visit, keep the comparison small. Pick candidates relevant to your task, then open their details and individual evaluation results.

Read the full configuration name. Model versions, reasoning settings, and providers can change the comparison. Record the date and full model name when sharing a result.

[ikaijua]

Reading the Intelligence Index

The Intelligence Index combines multiple evaluations using explicit weights. It is a summary of measured capability, not a universal percentage-correct score or a human IQ measurement.

• Check the version: included evaluations and weights change. Scores from different index versions are not directly interchangeable.
• Inspect individual results: similar overall scores can hide different strengths.
• Read the scoring method: tasks may use executable checks, model judges, or pairwise comparisons.
• Check coverage: the index is primarily text-based and English-language; multilingual and media evaluations are separate.

Use the aggregate to shortlist candidates, then inspect the tasks that resemble your work. A small score gap alone does not establish a consistent difference in everyday use.

Intelligence methodology and version history

[ikaijua]

Understanding cost and speed

A token is a unit of model input or output. Tokenizers and the amount generated for a task differ across models.

Metric	What it tells you
Input, output, and cache prices	Price per unit of usage
Cost per Task	Average cost for AA's measured workload
Output Speed	Tokens generated per second
Latency	Wait before the response starts
Time per Task	Task time under that page's definition

A lower token price can still produce a higher task bill if the model generates more tokens. Cache behavior also affects cost.

Timing definitions matter: language-model task time is estimated from token usage and representative output speed; coding-agent pages report active agent runtime. These are different measurements.

Definitions · API performance methodology

[ikaijua]

A practical model-selection workflow

For a research assistant, use the following workflow. This is a suggested selection process, not an AA benchmark result.

1. Define acceptance criteria: source accuracy, budget, turnaround time, and required output.
2. Shortlist by task: inspect relevant evaluations and retain candidates at different cost levels.
3. Read the capability–cost chart: check the axes. A Pareto-frontier point has no displayed alternative that improves both dimensions without a trade-off.
4. Test your own examples under comparable model and tool settings. Record failures, cost, time, and manual corrections.
5. Choose a default and a fallback: reserve expensive configurations for tasks that benefit from them.

Neither the top overall score nor the lowest price automatically makes the best choice. Publish the task sample, date, model version, and configuration alongside your selection.

[ikaijua]

Specialized leaderboards

Coding agents

The tested system combines a model with an execution harness: its tools, loop, and environment. Compare configurations, individual tasks, API cost, and runtime. An API-cost result is not a consumer subscription-plan comparison.

Coding-agent leaderboard · Methodology

Images and video

AA also uses blind human pairwise judgments, reported on an Elo scale. These are relative preferences within a benchmark, not quality percentages. Compare the relevant generation or editing task, and check settings such as resolution, duration, and audio.

Image methodology · Video methodology

Speech

Transcription, speech synthesis, and speech-to-speech interaction require different metrics. A language-model index cannot stand in for speech quality.

Speech evaluation methods

[ikaijua]

Arena comparison and common pitfalls

Your question	Useful starting point
Which response do users prefer?	Arena's relevant human-preference leaderboard
How capable, costly, and fast is a model?	AA's corresponding capability and API comparisons
Can an agent complete software work?	Agent evaluations plus your own task trials
Which image or video output suits you?	Relevant media leaderboards and samples

The distinction is not simply subjective versus objective. Most Arena leaderboards use human preferences, but Agent Arena has a different method. AA's media evaluations also use human judgments.

Do not interchange scores, assume a general index measures Chinese writing, or quote an undated screenshot. Keep model configuration and index version visible. Production workflows still need validation of tool use, authentication, long sessions, and recovery.

Arena methodology · Agent Arena · AA methodology

[ikaijua]

Access, data API, and attribution

Access

Public leaderboards are browsable. AA offers a free benchmark-data API and a commercial API with broader data.

The free API requires an Insights account and API key. Its documented limit is 1,000 requests per day. Commercial scope and access require contacting AA.

Integration

Call GET https://artificialanalysis.ai/api/v2/data/llms/models with an x-api-key header. Returned fields include evaluations, pricing, output speed, and latency.

Use stable model and creator IDs for joins. Keep keys server-side and cache responses. Free-API use requires attribution to Artificial Analysis and compliance with its terms.

Historical model names and values in example responses are examples, not current rankings.

API reference · Terms of use

[ikaijua]

Official references

• Company and team
• Model comparisons
• Methodology overview
• Intelligence Index methodology
• API performance methodology
• Coding-agent methodology
• Image methodology
• Video methodology
• Data API documentation
• Terms of use

Prices, quotas, model availability, and index versions change. Use the methodology attached to the relevant comparison.