feat: ABsmartly CLI and API client#2
Merged
Merged
Conversation
…ompatibility The parser's section key-value regex can't handle YAML list syntax in markdown body sections. Moving these to YAML frontmatter ensures gray-matter parses them as proper arrays, enabling template round-trips. Also adds comprehensive e2e test covering create -> export -> modify -> re-create.
Also fixes three bugs discovered during e2e testing: - parser: prevent \s* from matching newlines in key-value pattern, causing YAML list items to be merged into values - parser: add list parsing for YAML-style list fields in markdown body sections - builder: gracefully skip screenshot files that cannot be resolved instead of throwing
…t coverage Security: - Fix TOCTOU race in keyring credential file writes (mode 0o600) - Fix SSRF bypass via double-encoded paths in rawRequest - Validate HTTP methods in rawRequest instead of unsafe cast - Rename --show-key to --show-full-key with clear warning - Mask token display in config --detailed to last 4 chars - Add warning before --as-curl output containing API key - Set restrictive permissions on cache files Error handling: - Add try-catch to getPassword/deletePassword for keytar failures - Throw on corrupted credentials file instead of silent empty return - Log warning on screenshot image dimension detection failure - Add fetch timeout (30s) for screenshot URL resolution - Log auth errors in avatar fetch instead of silently skipping - Add 429 rate-limit retry support to axios adapter - Set process.exitCode=1 on partial failure in experiment commands Tests: - Add resolveExperimentId branching tests (7 scenarios) - Add updateExperiment merge-preserves-fields test - Add non-idempotent PUT retry exclusion test - Add 429 retry behavior test - Add network error (ECONNREFUSED) message test
- Use single scope 'mcp:access' (backend doesn't support space-separated)
- Use page.on('request') to capture OAuth code from redirect URL
- Bind callback server to 'localhost' instead of '127.0.0.1'
Prompts for old and new password instead of using admin reset endpoint. Uses PUT /auth/current-user with old_password + new_password.
Wraps POST /experiments/estimate/max_participants. Accepts unit type and applications by name or ID (resolved via listUnitTypes/listApplications), an optional --from date (defaults to 30d), and an optional --audience JSON filter. Prints a human-readable participant count summary by default; use -o json/yaml for the raw columnar response.
- Make --application optional (omit from payload when not provided, skip listApplications call entirely) - Validate --audience JSON client-side with a clear error message - Fix timestamp display: show field whenever column exists in response, use formatTimestampMs(0) -> 'N/A' instead of silent suppression - Warn explicitly when unit_count column is absent from response - Warn when API returns multiple rows (only first is shown) - Improve estimateMaxParticipants response validation: use createAPIError with response context, individual checks per field with key diagnostics - Fix printFormatted mock in tests to use top-level vi.mock factory - Add tests: no-application, ISO date --from, zero timestamp, missing unit_count column, multiple rows warning, invalid audience JSON, yaml output branch, exposure timestamp output lines
Adds tests in resources.test.ts following the same pattern as other resource tests: uses msw handlers in mock mode, hits the real API in live mode (USE_LIVE_API=1). Covers the happy path with/without applications, and an error-case test (skipIf live) for invalid response shape.
…s, screenshot labels, and various improvements - Add power matrix command to statistics with participant/MDE formatting and table output - Make group sequential fields conditional on analysis_type instead of always included - Support markdown image syntax with labels for variant screenshots - Convert APIError from interface to class with proper constructor - Consolidate branded type constructors into generic helper - Add new experiment creation options (MDE, baseline metric stats) - Improve OAuth token refresh, auth commands, and axios adapter - Add url utility, oauth-refresh tests, build-from-options tests, and live vitest runner - Expand test coverage for statistics, metrics, units, parser, serializer, and builder
6 tasks
This was referenced May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Complete CLI tool and API client for the ABsmartly A/B testing platform.
Core features
abs experiments list --state running | abs experiments stop | abs experiments archive-iinteractive prompts--rawflag — bypass all summarization and transformations--helpflagsPipe composition
--pass-throughforwards failed IDs for downstream retryQuality
Test plan
npx vitest run— 1202 passing, 0 failingnpx tsc --noEmit— cleanlist | stop | archiveverifiedUSE_LIVE_API=1roundtrip and client tests passabs experiments create -iinteractive modeabs experiments refresh-fieldson fresh profileSummary by CodeRabbit