Skip to content

browser-agent-v0.4.0

@guyernest guyernest tagged this 17 Nov 01:11
- ✨ Intelligent escalation from cheap to expensive methods
- ✨ DOM checks (free, ~10ms) → Playwright (~100ms) → Vision (~$0.01, ~1-2s)
- ✨ 40-75% cost reduction across all automation tasks
- ✨ 2-10x speed improvement over naive vision approach
- ✨ 85-95% success rate with free Playwright methods

- ✨ Direct Playwright + OpenAI vision integration
- ✨ Step types: navigate, click, fill, extract, wait_for_load_state, screenshot
- ✨ Click/fill with escalation chains (try Playwright first, vision fallback)
- ✨ Extract with automatic action execution (vision finds element, clicks it)
- ✨ Comprehensive [EXTRACT] logging shows vision API calls and responses

- ✨ BT Broadband Availability Checker (SSO login, address search, package extraction)
- ✨ Information Extraction (book catalog, news scraping, boolean checks)
- ✨ Form Filling (progressive field detection, validation)

- 🔧 BT Broadband: $0.04 → $0.01 per run (75% reduction)
- 🔧 Information Extraction: $0.05 → $0.03 per run (40% reduction)
- 🔧 Form Filling: $0.08 → $0.02 per run (75% reduction)

- 🔧 Triple wait for JavaScript redirects (SSO, SPA navigation)
- 🔧 Browser autofill synchronization
- 🔧 SPA form initialization delays
- 🔧 Modal overlay handling with force clicks

- 🔧 Validator accepts both starting_page (Nova Act) and starting_url (Progressive)
- 🔧 Auto-format detection via llm_provider field
- 🔧 Backward compatible with existing Nova Act scripts

- 📚 SCRIPT_BUILDER_GUIDE.md - 7 proven patterns, templates, best practices
- 📚 PROGRESSIVE_ESCALATION_STATUS.md - Architecture overview and metrics
- 📚 README_PROGRESSIVE_EXAMPLES.md - Migration guide, cost comparisons

- 📚 Standard click with escalation (copy-paste ready)
- 📚 JavaScript redirect wait pattern
- 📚 Form filling with progressive detection

| Example | Steps | Vision Calls | Playwright Ops | Cost | Speed |
|---------|-------|--------------|----------------|------|-------|
| BT Broadband | 34 | 1-2 | 8-10 | $0.01 | ~25s |
| Info Extraction | 15 | 3 | 1 | $0.03 | ~8s |
| Form Filling | 18 | 3 | 7 | $0.02 | ~6s |

None - fully backward compatible with v0.3.x

- ✅ BT Broadband full flow (login, search, address selection, package extraction)
- ✅ Dual format validation (Nova Act + Progressive Escalation)
- ✅ Vision logging and action execution
- ✅ SSO redirect handling
- ✅ Form autofill timing
- ✅ Modal overlay force clicks

Existing Nova Act scripts continue to work. To use progressive escalation:
1. Add "llm_provider": "openai" to script
2. Change "starting_page" to "starting_url"
3. Replace "action: act" steps with progressive escalation patterns (see SCRIPT_BUILDER_GUIDE.md)

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Assets 2
Loading