v0.2.0-beta.3

Automation Sandbox v0.2.0-beta.3

Preview NuGet package artifact release.

Breaking Changes

Both changes are in the LLM fallback path. Consumers who never configure an LLM provider are unaffected.

• A single LLM provider's pick is no longer accepted (#10). Acceptance is now by consensus: at least SimilarityWeights.MinimumConsensusVotes (default 2) providers must independently name the same candidate. Self-reported confidence is still recorded but is never compared or thresholded, because those numbers are not calibrated across model architectures. This fails quietly by design - with one provider configured, healing degrades to the heuristic result instead of raising an error. If you relied on single-provider LLM healing, configure a second provider from a different model family, or expect heuristic-only behaviour. MinimumLlmConfidence remains on SimilarityWeights and is still reported, but no longer gates anything.
• Worst-case provider latency rose from ~15s to ~35s (#11). Providers now retry transient failures, so a slow or failing endpoint occupies its per-attempt timeout more than once. The new TotalTimeout (35s cloud, 70s Ollama) caps the whole operation including backoff. Set maxRetries: 0 to restore single-shot behaviour.

Highlights

• Opt-in joint locator reconciliation (#144): ResolveBatch / ResolveBatchAsync reconcile independently accepted top claims against one shared tree without changing the existing single-locator APIs. Snapshot-local candidate identity supports empty or duplicate AutomationId values; ambiguous ownership declines every claimant and runner-ups are never promoted. This is a collision guard, not an absence detector: uncontested deleted-element false heals remain unchanged.
• Consensus acceptance for LLM picks (#10): Providers are polled in parallel and a pick is accepted only on agreement. Invalid candidate ids are discarded before the vote is counted, so one hallucinating provider cannot invalidate everyone else's valid votes. A three-way split and a tie for the lead both count as disagreement and fall back to the heuristic result. HealResult.AgreedProviders records who agreed.
• Provider resilience (#11): Retry with exponential backoff and jitter on transient failures (429, 500, 502, 503, 504, HttpRequestException); immediate fail-fast on 400/401/403/404. Retry-After is honoured up to a 10s ceiling - a longer value is read as quota exhaustion and fails fast rather than stalling a test run. Dual timeout budgets: per attempt and per operation.
• Per-provider attempt telemetry (#11): LlmHealingResult.AttemptCount and HealResult.ProviderAttempts record how many attempts each provider needed, so a flaky provider is identifiable rather than merely producing a failed heal.
• Complete resolution telemetry and healing report schema v8 (#82, #144): every resolution attempt records an explicit outcome, including accepted, accepted-unverified, retry-failed, ambiguous, ownership-conflict, low-evidence, low-confidence, no-candidates, no-consensus, provider-error, and unspecified. Batch entries may also carry snapshot-local candidate identity and reconciliation disposition. HealingReportDocument.AcceptedEvents retains the accepted-only compatibility view; older reports upgrade with new fields left null, meaning "not recorded by that build" rather than "there were none".
• Gating live consensus check (#84): the nightly workflow now requires independent Groq and Mistral providers to agree on the shortlist-valid ground-truth candidate in a known scenario. The result and both provider names are written to the GitHub Step Summary; broader nightly evaluation remains collection-only telemetry. The release workflow itself is deliberately not coupled to third-party API availability.
• Configurable provider names (#10): every provider takes a name: parameter. OpenAiHealingProvider accepts any OpenAI-compatible endpoint, so several instances (for example Groq and Cerebras) can vote alongside each other and remain distinguishable in the report.
• Shared HTTP provider base (#48): the four providers now derive from HttpLlmHealingProvider, which owns orchestration, validation and retry wiring. Behaviour-neutral; public constructors are unchanged.

Known Limitations

• Calibration: Real-world HandBrake and ShareX measurements show that higher confidence thresholds reduce false heals, but the magnitude does not transfer cleanly between applications. The shipped MinimumConfidence remains 0.50; applications should calibrate against their own UI history rather than treating the measured 0.75-0.80 range as portable.
• Accuracy: The opt-in batch ownership guard eliminated all observed shared-candidate collisions while preserving all 79 correct survivor heals in the frozen HandBrake/ShareX study. It is not an absence detector: 15 uncontested removed-element false heals remained unchanged.
• LLM Providers: Provider integrations are covered by mocked-HTTP unit tests, while the nightly workflow gates a known ground-truth consensus scenario against independent Groq and Mistral providers. This still depends on third-party availability and free-tier quota, so release creation itself does not run the live gate. llm-smoke.yml remains a manual single-endpoint diagnostic after GitHub Models began returning HTTP 410 ahead of retirement (#21, #44).
• Platform: Desktop automation is Windows-only (FlaUI / UIA3). The core heuristic engine and web automation path are cross-platform and verified on Linux in CI (#13).

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery
• AutomationSandbox.IntentAutomation
• AutomationSandbox.PlaywrightLiveExploration

The packages are available both as GitHub Release assets and from nuget.org, published through Trusted Publishing (OIDC, without a stored API key).