Magnitude: The open source, AI-native testing framework for web apps

---

End-to-end testing framework powered by visual AI agents that see your interface and adapt to any changes in it.

How it works

• ✍️ Build test cases easily with natural language
• 🧠 Strong reasoning agent to plan and adjust tests
• 👁️ Fast visual agent to reliably execute runs
• 📄 Plan is saved to execute runs the same way
• 🛠 Reasoning agent steps in if there is a problem
• 🏃‍♂️ Run tests locally or in CI/CD pipelines

↕️ Magnitude test case in action! ↕️

test('can add and complete todos', { url: 'https://magnitodo.com' }, async (agent) => {
    await agent.act('create 3 todos', {
        data: 'Take out the trash, Buy groceries, Build more test cases with Magnitude'
    });
    await agent.check('should see all 3 todos');
    await agent.act('mark each todo complete');
    await agent.check('says 0 items left');
});

Setup

Install Magnitude

1. Install our test runner in the node project you want to test (or see our demo repo if you don't have a project to try it on)

npm install --save-dev magnitude-test

2. Setup Magnitude in your project by running:

npx magnitude init

This will create a basic tests directory tests/magnitude with:

• magnitude.config.ts: Magnitude test configuration file
• example.mag.ts: An example test file

3. Configure an LLM

The easiest way to set up an LLM for Magnitude is to set the ANTHROPIC_API_KEY environment variable. Claude Sonnet 4 will be used by default. See docs for more details.

🚀 Now you're ready to run tests!

Running tests

Run your Magnitude tests with:

npx magnitude

This will run all Magnitude test files discovered with the *.mag.ts pattern. If the agent finds a problem with your app, it will tell you what happened and describe the bug!

To run many tests in parallel, add -w <workers>

Building test cases

Now that you've got Magnitude set up, you can create real test cases for your app. Here's an example for a general idea:

import { test } from 'magnitude-test';

test('can log in and create company', async (agent) => {
    await agent.act('Log in to the app', {
        data: { username: 'test-user@magnitude.run', password: 'test' }
    });
    await agent.check('Can see dashboard');
    await agent.act('Create a new company', { data: 'Make up the first 2 values and use defaults for the rest' });
    await agent.check('Company added successfully');
});

Act, checks, and data are all natural language. Think of it like you're describing how to test a particular flow to a co-worker - what steps they need to take, what they should check for, and what test data to use.

For more information on how to build test cases see our docs.

Integrating with CI/CD

You can run Magnitude tests in CI anywhere that you could run Playwright tests, just include LLM client credentials. For instructions on running tests cases on GitHub actions, see here.

Contact

To get a personalized demo or see how Magnitude can help your company, feel free to reach out to us at founders@magnitude.run

You can also join our Discord community for help or any suggestions!