AXS v0.1 — Every agent benchmark measures how good the agent is. None measure how good the service is for agents.

[sat3n]

I read Mathias Biilmann’s “Introducing AX”. Spot on but I noticed a gap.

Every agent benchmark measures how good the agent is.

None of them measure how good the service is for agents. So, I built a framework to fill it.

AXS (Agent Experience Score) is an open-source scoring framework that measures service-side Agent Experience across seven dimensions, scored 0 to 100: discoverability, schema quality, reliability, recoverability, latency, auth, and determinism.
Objective. Reproducible. Protocol-neutral. Apache 2.0.

This builds on the Agent Experience (AX) movement that @mathias Biilmann at Netlify started in January 2025. The community has established what good AX looks like. What was missing is a standardised way to measure it. AXS is that measurement layer.

The framework is live on GitHub: https://github.com/sat3n/axs-framework

First benchmark results (10 to 20 services scored) coming in the next two weeks(approx.).

I built this with Claude and Grok as thinking partners. The full story of how is in the Substack post here : https://satenderkundu.substack.com/p/ax-has-a-measurement-problem-heres.

I would be grateful for any thoughts from this community:

Star the repo and try the test harness on your own services
Submit your scores and category weights via PR if you like
Tell me where it breaks or how to improve it

Happy to make AX measurable together.

Satender (@sat3n)