Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ vizzly run "npm test" --parallel-id "ci-run-123" # For parallel CI builds

**Processing Options:**
- `--wait` - Wait for build completion and exit with appropriate code
- `--threshold <number>` - Comparison threshold (0-1, default: 0.01)
- `--threshold <number>` - Comparison threshold (CIEDE2000 Delta E, default: 2.0)
- `--upload-timeout <ms>` - Upload wait timeout in ms
- `--upload-all` - Upload all screenshots without SHA deduplication

Expand Down Expand Up @@ -279,7 +279,7 @@ vizzly dev stop
**Dev Command Options:**
- `--set-baseline` - Accept current screenshots as new baseline
- `--baseline-build <id>` - Use specific build as baseline (requires API token)
- `--threshold <number>` - Comparison threshold (0-1, default: 0.1)
- `--threshold <number>` - Comparison threshold (CIEDE2000 Delta E, default: 2.0)
- `--port <port>` - Server port (default: 47392)
- `--timeout <ms>` - Server timeout (default: 30000)
- `--open` - Auto-open dashboard in browser (start command only)
Expand Down Expand Up @@ -357,9 +357,9 @@ export default {
timeout: 30000
},

// Comparison configuration
// Comparison configuration (CIEDE2000 Delta E: 0=exact, 1=JND, 2=recommended)
comparison: {
threshold: 0.1,
threshold: 2.0,
ignoreAntialiasing: true
},

Expand Down
4 changes: 2 additions & 2 deletions SDK-API.md
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,7 @@ Retrieves detailed information about a specific comparison, including diff image
"name": "homepage-desktop-chrome",
"status": "completed",
"diff_percentage": 2.5,
"threshold": 0.1,
"threshold": 2.0,
"has_diff": true,
"current_name": "homepage-desktop-chrome",
"current_browser": "chrome",
Expand Down Expand Up @@ -467,7 +467,7 @@ Search for comparisons by screenshot name across builds. Useful for debugging vi
"name": "homepage-desktop-chrome",
"status": "completed",
"diff_percentage": 2.5,
"threshold": 0.1,
"threshold": 2.0,
"has_diff": true,
"build_id": "build-uuid",
"build_name": "CLI Build",
Expand Down
4 changes: 2 additions & 2 deletions claude-plugin/skills/debug-visual-regression/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ Response:
"name": "homepage",
"status": "failed",
"diffPercentage": 2.3,
"threshold": 0.1,
"threshold": 2.0,
"mode": "local",
"baselinePath": "/Users/you/project/.vizzly/baselines/homepage.png",
"currentPath": "/Users/you/project/.vizzly/screenshots/homepage.png"
Expand Down Expand Up @@ -170,7 +170,7 @@ Response:
"name": "login-page",
"status": "failed",
"diffPercentage": 0.8,
"threshold": 0.1,
"threshold": 2.0,
"mode": "cloud",
"baselineUrl": "https://app.vizzly.dev/screenshots/abc123/baseline.png",
"currentUrl": "https://app.vizzly.dev/screenshots/abc123/current.png",
Expand Down
10 changes: 5 additions & 5 deletions docs/api-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Capture a screenshot for visual regression testing.
```javascript
{
// Comparison settings
threshold: 0.01, // Pixel difference threshold (0-1)
threshold: 2.0, // CIEDE2000 Delta E (0=exact, 1=JND, 2=recommended)

// Metadata for organization (all optional)
properties: {
Expand Down Expand Up @@ -451,7 +451,7 @@ Upload screenshots from a directory.
- `--commit <sha>` - Git commit SHA override
- `--message <msg>` - Commit message override
- `--environment <env>` - Environment name (default: "test")
- `--threshold <number>` - Comparison threshold (0-1)
- `--threshold <number>` - Comparison threshold (CIEDE2000 Delta E)
- `--token <token>` - API token override
- `--wait` - Wait for build completion
- `--upload-all` - Upload all screenshots without SHA deduplication
Expand Down Expand Up @@ -484,7 +484,7 @@ Run tests with Vizzly integration.

*Processing Options:*
- `--wait` - Wait for build completion and exit with appropriate code
- `--threshold <number>` - Comparison threshold (0-1, default: 0.01)
- `--threshold <number>` - Comparison threshold (CIEDE2000 Delta E, default: 2.0)
- `--upload-timeout <ms>` - Upload wait timeout in ms (default: from config or 30000)
- `--upload-all` - Upload all screenshots without SHA deduplication

Expand Down Expand Up @@ -528,7 +528,7 @@ Run tests in TDD mode with local visual comparisons.
*Build Configuration:*
- `--branch <branch>` - Git branch override
- `--environment <env>` - Environment name (default: "test")
- `--threshold <number>` - Comparison threshold (0-1, default: 0.1)
- `--threshold <number>` - Comparison threshold (CIEDE2000 Delta E, default: 2.0)
- `--token <token>` - API token override

**Behavior:**
Expand Down Expand Up @@ -661,7 +661,7 @@ Configuration loaded via cosmiconfig in this order:

// Comparison Configuration
comparison: {
threshold: number // Pixel difference threshold (default: 0.1)
threshold: number // CIEDE2000 Delta E (default: 2.0)
}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/doctor-command.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The `doctor` command provides a quick preflight to validate your local setup. It

- Node.js version: must be 20 or newer
- `apiUrl`: must be a valid `http` or `https` URL
- `comparison.threshold`: a number between 0 and 1
- `comparison.threshold`: a non-negative number (CIEDE2000 Delta E units, default: 2.0)
- Effective `server.port`: reports the port without binding (default `47392`)

## Usage
Expand Down
6 changes: 3 additions & 3 deletions docs/tdd-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ Starts the dashboard server in the background as a detached process and returns

Options:
- `--port <port>` - Server port (default: 47392)
- `--threshold <number>` - Comparison threshold (default: 0.1)
- `--threshold <number>` - Comparison threshold in CIEDE2000 Delta E (default: 2.0)
- `--baseline-build <id>` - Use specific build as baseline
- `--open` - Auto-open dashboard in browser

Expand All @@ -236,7 +236,7 @@ vizzly tdd run "npm test" [options]
Options:
- `--set-baseline` - Accept screenshots as new baseline
- `--port <port>` - Server port (default: 47392)
- `--threshold <number>` - Comparison threshold (default: 0.1)
- `--threshold <number>` - Comparison threshold in CIEDE2000 Delta E (default: 2.0)
- `--baseline-build <id>` - Use specific build as baseline
- `--timeout <ms>` - Server timeout (default: 30000)

Expand Down Expand Up @@ -358,7 +358,7 @@ Configure in `vizzly.config.js`:
```javascript
export default {
comparison: {
threshold: 0.01 // 1% difference tolerance
threshold: 2.0 // CIEDE2000 Delta E (0=exact, 1=JND, 2=recommended)
}
};
```
Expand Down
12 changes: 6 additions & 6 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@
"registry": "https://registry.npmjs.org/"
},
"dependencies": {
"@vizzly-testing/honeydiff": "^0.4.3",
"@vizzly-testing/honeydiff": "^0.5.0",
"commander": "^14.0.0",
"cosmiconfig": "^9.0.0",
"dotenv": "^17.2.1",
Expand Down
8 changes: 4 additions & 4 deletions src/commands/doctor.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,14 @@ export async function doctorCommand(options = {}, globalOptions = {}) {
// Validate threshold (0..1 inclusive)
let threshold = Number(config?.comparison?.threshold);
diagnostics.configuration.threshold = threshold;
let thresholdValid =
Number.isFinite(threshold) && threshold >= 0 && threshold <= 1;
// CIEDE2000 threshold: 0 = exact, 1 = JND, 2 = recommended, 3+ = permissive
let thresholdValid = Number.isFinite(threshold) && threshold >= 0;
diagnostics.configuration.thresholdValid = thresholdValid;
if (thresholdValid) {
output.success(`Threshold: ${threshold}`);
output.success(`Threshold: ${threshold} (CIEDE2000 Delta E)`);
} else {
hasErrors = true;
output.error('Invalid threshold (expected number between 0 and 1)');
output.error('Invalid threshold (expected non-negative number)');
}

// Report effective port without binding
Expand Down
4 changes: 2 additions & 2 deletions src/commands/init.js
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,9 @@ export class InitCommand {
timeout: 30000
},

// Comparison configuration
// Comparison configuration (CIEDE2000 Delta E: 0=exact, 1=JND, 2=recommended)
comparison: {
threshold: 0.1
threshold: 2.0
},

// TDD configuration
Expand Down
6 changes: 4 additions & 2 deletions src/commands/tdd.js
Original file line number Diff line number Diff line change
Expand Up @@ -231,8 +231,10 @@ export function validateTddOptions(testCommand, options) {

if (options.threshold !== undefined) {
let threshold = parseFloat(options.threshold);
if (isNaN(threshold) || threshold < 0 || threshold > 1) {
errors.push('Threshold must be a number between 0 and 1');
if (isNaN(threshold) || threshold < 0) {
errors.push(
'Threshold must be a non-negative number (CIEDE2000 Delta E)'
);
}
}

Expand Down
6 changes: 4 additions & 2 deletions src/commands/upload.js
Original file line number Diff line number Diff line change
Expand Up @@ -250,8 +250,10 @@ export function validateUploadOptions(screenshotsPath, options) {

if (options.threshold !== undefined) {
let threshold = parseFloat(options.threshold);
if (isNaN(threshold) || threshold < 0 || threshold > 1) {
errors.push('Threshold must be a number between 0 and 1');
if (isNaN(threshold) || threshold < 0) {
errors.push(
'Threshold must be a non-negative number (CIEDE2000 Delta E)'
);
}
}

Expand Down
2 changes: 1 addition & 1 deletion src/reporter/src/components/views/settings-view.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import {

function getInitialFormData(config) {
return {
threshold: config?.comparison?.threshold ?? 0.1,
threshold: config?.comparison?.threshold ?? 2.0,
port: config?.server?.port ?? 47392,
timeout: config?.server?.timeout ?? 30000,
buildName: config?.build?.name ?? 'Build {timestamp}',
Expand Down
2 changes: 1 addition & 1 deletion src/server/handlers/tdd-handler.js
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ export const createTddHandler = (
}

// Unwrap double-nested properties if needed (client SDK wraps options in properties field)
// This happens when test helper passes { properties: {...}, threshold: 0.1 }
// This happens when test helper passes { properties: {...}, threshold: 2.0 }
// and client SDK wraps it as { properties: options }
let unwrappedProperties = properties;
if (properties.properties && typeof properties.properties === 'object') {
Expand Down
2 changes: 1 addition & 1 deletion src/services/config-service.js
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ export class ConfigService {
batchSize: 10,
timeout: 30000,
},
comparison: { threshold: 0.1 },
comparison: { threshold: 2.0 },
tdd: { openReport: false },
plugins: [],
};
Expand Down
4 changes: 2 additions & 2 deletions src/services/tdd-service.js
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ export class TddService {
this.diffPath = safePath(this.workingDir, '.vizzly', 'diffs');
this.baselineData = null;
this.comparisons = [];
this.threshold = config.comparison?.threshold || 0.1;
this.threshold = config.comparison?.threshold || 2.0;
this.signatureProperties = []; // Custom properties from project's baseline_signature_properties

// Check if we're in baseline update mode
Expand Down Expand Up @@ -1226,7 +1226,7 @@ export class TddService {
try {
// Try to compare - honeydiff will throw if dimensions don't match
const result = await compare(baselineImagePath, currentImagePath, {
colorThreshold: this.threshold, // YIQ color threshold (0.0-1.0), default 0.1
threshold: this.threshold, // CIEDE2000 Delta E (2.0 = recommended default)
antialiasing: true,
diffPath: diffImagePath,
overwrite: true,
Expand Down
2 changes: 1 addition & 1 deletion src/types/config.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ export { VizzlyConfig } from './index';
* port: 47392
* },
* comparison: {
* threshold: 0.1
* threshold: 2.0 // CIEDE2000 Delta E (0=exact, 1=JND, 2=recommended)
* }
* });
*/
Expand Down
4 changes: 2 additions & 2 deletions src/utils/config-loader.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ let DEFAULT_CONFIG = {
timeout: 30000,
},

// Comparison Configuration
// Comparison Configuration (CIEDE2000 Delta E: 0=exact, 1=JND, 2=recommended)
comparison: {
threshold: 0.1,
threshold: 2.0,
},

// TDD Configuration
Expand Down
7 changes: 4 additions & 3 deletions src/utils/config-schema.js
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,10 @@ let uploadSchema = z.object({

/**
* Comparison configuration schema
* threshold: CIEDE2000 Delta E units (0.0 = exact, 1.0 = JND, 2.0 = recommended, 3.0+ = permissive)
*/
let comparisonSchema = z.object({
threshold: z.number().min(0).max(1).default(0.1),
threshold: z.number().min(0).default(2.0),
});

/**
Expand Down Expand Up @@ -68,7 +69,7 @@ export let vizzlyConfigSchema = z
batchSize: 10,
timeout: 30000,
}),
comparison: comparisonSchema.default({ threshold: 0.1 }),
comparison: comparisonSchema.default({ threshold: 2.0 }),
tdd: tddSchema.default({ openReport: false }),
plugins: z.array(z.string()).default([]),

Expand All @@ -85,7 +86,7 @@ export let vizzlyConfigSchema = z
server: { port: 47392, timeout: 30000 },
build: { name: 'Build {timestamp}', environment: 'test' },
upload: { screenshotsDir: './screenshots', batchSize: 10, timeout: 30000 },
comparison: { threshold: 0.1 },
comparison: { threshold: 2.0 },
tdd: { openReport: false },
plugins: [],
});
Expand Down
6 changes: 3 additions & 3 deletions test-d/config.test-d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ expectType<VizzlyConfig>(
timeout: 30000,
},
comparison: {
threshold: 0.1,
threshold: 2.0,
},
tdd: {
openReport: false,
Expand All @@ -50,7 +50,7 @@ expectType<VizzlyConfig>(
defineConfig({
apiKey: process.env.VIZZLY_TOKEN,
comparison: {
threshold: 0.05,
threshold: 1.5,
},
})
);
Expand Down Expand Up @@ -85,7 +85,7 @@ let typicalConfig = defineConfig({
port: 47392,
},
comparison: {
threshold: 0.1,
threshold: 2.0,
},
});

Expand Down
Loading