v0.2.0-beta.4

Automation Sandbox v0.2.0-beta.4

Preview NuGet package artifact release. Rolls up the work merged since v0.2.0-beta.3.

Breaking Changes

• SimilarityWeights.MinimumLlmConfidence has been removed (#253). It stopped gating LLM acceptance when the independent-agreement quorum landed (v0.2.0-beta.3), but stayed on the public configuration surface as a no-op. Code that sets it (new SimilarityWeights { MinimumLlmConfidence = ... }) no longer compiles; delete the assignment. LLM acceptance is controlled by MinimumConsensusVotes only. Self-reported confidence is still recorded for audit, never compared or thresholded.

Highlights

• Healing modes (#248): SelfHealingEngine takes an explicit HealingMode - Observe (record only), Review (propose, never apply), AutoHeal (apply and retry), FailClosed (never heal, rethrow immediately). The default is Review, so a heal is never silently applied unless you ask for it.
• Threshold profiles and per-application calibration (#252): Conservative / Balanced / Aggressive presets for SimilarityWeights, plus a calibrate CLI command that sweeps a repository's own locator history and reports the false-heal / recall trade-off at each threshold. The shipped MinimumConfidence stays 0.50; calibrate against your own UI rather than treating a measured range as portable.
• Secret and PII redaction on by default (#249, #202, #270): DOM text and TestIntent strings are scrubbed of API keys and secrets (Stripe, GCP service-account keys, generic bearer tokens) before any of it reaches an LLM provider. The TextSanitizer hook is opt-in for custom rules on top.
• Prompt-injection hardening (#250): LlmHealingPrompt isolates candidate element data from the instruction envelope, so injected directives inside element text cannot steer the model's pick.
• xUnit and NUnit fixture helpers (#262, #283): SelfHealing.Testing.SelfHealingTestBase drops self-healing into a consumer test suite with no per-test wiring, verified end-to-end under both runners.
• Playwright end-to-end sample (#251): a runnable consumer showing live web discovery, intent-driven test generation, self-healing, and the safety guardrails together.
• DORA and engine-reliability metrics workflow (#323, #328): a weekly (and manual) PowerShell report - deployment frequency, lead time for changes, change-failure rate, MTTR - alongside the self-healing gate thresholds read straight from SimilarityWeights.cs. Visibility only; it gates nothing, and it fails loudly rather than publishing a report built on an empty API fetch.
• Per-package CycloneDX SBOMs (#274): the release workflow generates and attaches one SBOM per package with a pinned tool version.
• Beta-exit criteria and public API stability policy (#255, #282): documented Tier 1/2/3 API surface and the bar for a 1.0.
• 10,000-control resolver benchmark (#265, #290) and a custom ILlmHealingProvider implementation guide (#263, #284).

Fixes

• Batch reconciliation: mixed heuristic / LLM contention on the same candidate now resolves as ambiguous (declining every claimant) instead of arbitrarily by score scale (#268).
• LocatorRepository freshness is keyed on a content hash, not just mtime/size, so an in-place edit of the same length is picked up (#273). Harness contention detection is keyed on path-qualified element identity, not the AutomationId string, closing the empty-id and duplicate-id blind spots (#272, #292).
• LLM response parsing no longer discards a valid consensus vote over a malformed optional field: a string-valued confidence, a numeric candidateId (#304), or a non-string reasoning (#321) are all parsed defensively.
• Resource leaks on error paths closed: ApplicationConnector.Launch no longer orphans the launched process if UIA initialisation throws (#305, and Attach is left to dispose only, never kill); PlaywrightLiveExplorer.DisposeAsync runs _playwright.Dispose() in a finally so a browser-close fault cannot leak the driver (#306).
• Generated FlaUI tests compile: the file-upload path emits fully-qualified FlaUI.Core.Definitions.ControlType (#303).
• Web mapping: <form> maps to Group to match its ARIA role (#308); a symbol-only ARIA role no longer throws in PlaywrightLocatorEmitter (#309); ARIA role normalisation is aligned with the UIA ControlType taxonomy (#267).
• Intent scoring: a 2-character element label no longer earns a spurious exact-match bonus from a bare substring hit (#307).
• Healing report HTML is written atomically (temp file then rename), matching the JSON path (#310).
• Codegen: regex strings are escaped for C# and TypeScript UrlContains output (#266).
• Windows net48 coverage is restored (coverlet 8.0+ ships no .NET Framework collector) and guarded so the Discovery assembly cannot silently drop out again (#289).
• Dead code removed from TreeCalibrator (orphaned reference-identity mutation helpers, an unused probe kind) and PlaywrightApplicationConnector (#311).
• Documentation realigned with the code: HealingHistory schema fields, M6 implementation status, versioning tiers, the AGENTS.md project/workflow inventory, stale model defaults and self-contradicting benchmark figures, and English/Turkish parity (#300, #312, #313, #264).
• Dependency upgrades across the libraries and the test runner (#294).

Known Limitations

• Calibration: higher confidence thresholds reduce false heals, but the magnitude does not transfer between applications. Calibrate against your own UI history; the shipped MinimumConfidence is 0.50.
• Accuracy: the opt-in batch ownership guard eliminates observed shared-candidate collisions while preserving correct survivor heals, but it is a collision guard, not an absence detector - uncontested deleted-element false heals are unchanged.
• LLM providers: covered by mocked-HTTP unit tests; the nightly workflow gates one known ground-truth agreement scenario against independent providers. Release creation does not run a live gate. llm-smoke.yml is a manual single-endpoint diagnostic after GitHub Models' retirement (#44).
• Platform: desktop automation is Windows-only (FlaUI / UIA3). The heuristic engine and the web path are cross-platform and verified on Linux in CI. Linux desktop discovery via AT-SPI is tracked in #17.

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery
• AutomationSandbox.IntentAutomation
• AutomationSandbox.PlaywrightLiveExploration

This release publishes package files as GitHub Release assets. They are also pushed to nuget.org via Trusted Publishing.

v0.2.0-beta.3

Automation Sandbox v0.2.0-beta.3

Preview NuGet package artifact release.

Breaking Changes

Both changes are in the LLM fallback path. Consumers who never configure an LLM provider are unaffected.

• A single LLM provider's pick is no longer accepted (#10). Acceptance is now by consensus: at least SimilarityWeights.MinimumConsensusVotes (default 2) providers must independently name the same candidate. Self-reported confidence is still recorded but is never compared or thresholded, because those numbers are not calibrated across model architectures. This fails quietly by design - with one provider configured, healing degrades to the heuristic result instead of raising an error. If you relied on single-provider LLM healing, configure a second provider from a different model family, or expect heuristic-only behaviour. MinimumLlmConfidence remains on SimilarityWeights and is still reported, but no longer gates anything.
• Worst-case provider latency rose from ~15s to ~35s (#11). Providers now retry transient failures, so a slow or failing endpoint occupies its per-attempt timeout more than once. The new TotalTimeout (35s cloud, 70s Ollama) caps the whole operation including backoff. Set maxRetries: 0 to restore single-shot behaviour.

Highlights

• Opt-in joint locator reconciliation (#144): ResolveBatch / ResolveBatchAsync reconcile independently accepted top claims against one shared tree without changing the existing single-locator APIs. Snapshot-local candidate identity supports empty or duplicate AutomationId values; ambiguous ownership declines every claimant and runner-ups are never promoted. This is a collision guard, not an absence detector: uncontested deleted-element false heals remain unchanged.
• Consensus acceptance for LLM picks (#10): Providers are polled in parallel and a pick is accepted only on agreement. Invalid candidate ids are discarded before the vote is counted, so one hallucinating provider cannot invalidate everyone else's valid votes. A three-way split and a tie for the lead both count as disagreement and fall back to the heuristic result. HealResult.AgreedProviders records who agreed.
• Provider resilience (#11): Retry with exponential backoff and jitter on transient failures (429, 500, 502, 503, 504, HttpRequestException); immediate fail-fast on 400/401/403/404. Retry-After is honoured up to a 10s ceiling - a longer value is read as quota exhaustion and fails fast rather than stalling a test run. Dual timeout budgets: per attempt and per operation.
• Per-provider attempt telemetry (#11): LlmHealingResult.AttemptCount and HealResult.ProviderAttempts record how many attempts each provider needed, so a flaky provider is identifiable rather than merely producing a failed heal.
• Complete resolution telemetry and healing report schema v8 (#82, #144): every resolution attempt records an explicit outcome, including accepted, accepted-unverified, retry-failed, ambiguous, ownership-conflict, low-evidence, low-confidence, no-candidates, no-consensus, provider-error, and unspecified. Batch entries may also carry snapshot-local candidate identity and reconciliation disposition. HealingReportDocument.AcceptedEvents retains the accepted-only compatibility view; older reports upgrade with new fields left null, meaning "not recorded by that build" rather than "there were none".
• Gating live consensus check (#84): the nightly workflow now requires independent Groq and Mistral providers to agree on the shortlist-valid ground-truth candidate in a known scenario. The result and both provider names are written to the GitHub Step Summary; broader nightly evaluation remains collection-only telemetry. The release workflow itself is deliberately not coupled to third-party API availability.
• Configurable provider names (#10): every provider takes a name: parameter. OpenAiHealingProvider accepts any OpenAI-compatible endpoint, so several instances (for example Groq and Cerebras) can vote alongside each other and remain distinguishable in the report.
• Shared HTTP provider base (#48): the four providers now derive from HttpLlmHealingProvider, which owns orchestration, validation and retry wiring. Behaviour-neutral; public constructors are unchanged.

Known Limitations

• Calibration: Real-world HandBrake and ShareX measurements show that higher confidence thresholds reduce false heals, but the magnitude does not transfer cleanly between applications. The shipped MinimumConfidence remains 0.50; applications should calibrate against their own UI history rather than treating the measured 0.75-0.80 range as portable.
• Accuracy: The opt-in batch ownership guard eliminated all observed shared-candidate collisions while preserving all 79 correct survivor heals in the frozen HandBrake/ShareX study. It is not an absence detector: 15 uncontested removed-element false heals remained unchanged.
• LLM Providers: Provider integrations are covered by mocked-HTTP unit tests, while the nightly workflow gates a known ground-truth consensus scenario against independent Groq and Mistral providers. This still depends on third-party availability and free-tier quota, so release creation itself does not run the live gate. llm-smoke.yml remains a manual single-endpoint diagnostic after GitHub Models began returning HTTP 410 ahead of retirement (#21, #44).
• Platform: Desktop automation is Windows-only (FlaUI / UIA3). The core heuristic engine and web automation path are cross-platform and verified on Linux in CI (#13).

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery
• AutomationSandbox.IntentAutomation
• AutomationSandbox.PlaywrightLiveExploration

The packages are available both as GitHub Release assets and from nuget.org, published through Trusted Publishing (OIDC, without a stored API key).

v0.2.0-beta.2

Automation Sandbox v0.2.0-beta.2

Preview NuGet package artifact release.

Breaking Changes

Both changes are in the LLM fallback path. Consumers who never configure an LLM provider are unaffected.

• A single LLM provider's pick is no longer accepted (#10). Acceptance is now by consensus: at least SimilarityWeights.MinimumConsensusVotes (default 2) providers must independently name the same candidate. Self-reported confidence is still recorded but is never compared or thresholded, because those numbers are not calibrated across model architectures. This fails quietly by design - with one provider configured, healing degrades to the heuristic result instead of raising an error. If you relied on single-provider LLM healing, configure a second provider from a different model family, or expect heuristic-only behaviour. MinimumLlmConfidence remains on SimilarityWeights and is still reported, but no longer gates anything.
• Worst-case provider latency rose from ~15s to ~35s (#11). Providers now retry transient failures, so a slow or failing endpoint occupies its per-attempt timeout more than once. The new TotalTimeout (35s cloud, 70s Ollama) caps the whole operation including backoff. Set maxRetries: 0 to restore single-shot behaviour.

Highlights

• Consensus acceptance for LLM picks (#10): Providers are polled in parallel and a pick is accepted only on agreement. Invalid candidate ids are discarded before the vote is counted, so one hallucinating provider cannot invalidate everyone else's valid votes. A three-way split and a tie for the lead both count as disagreement and fall back to the heuristic result. HealResult.AgreedProviders records who agreed.
• Provider resilience (#11): Retry with exponential backoff and jitter on transient failures (429, 500, 502, 503, 504, HttpRequestException); immediate fail-fast on 400/401/403/404. Retry-After is honoured up to a 10s ceiling - a longer value is read as quota exhaustion and fails fast rather than stalling a test run. Dual timeout budgets: per attempt and per operation.
• Per-provider attempt telemetry (#11): LlmHealingResult.AttemptCount and HealResult.ProviderAttempts record how many attempts each provider needed, so a flaky provider is identifiable rather than merely producing a failed heal.
• Healing report schema v6: adds AgreedProviders (v5) and ProviderAttempts (v6). Older reports upgrade in place on the next write; the new fields stay null on upgraded entries, which means "not recorded by that build" rather than "there were none".
• Configurable provider names (#10): every provider takes a name: parameter. OpenAiHealingProvider accepts any OpenAI-compatible endpoint, so several instances (for example Groq and Cerebras) can vote alongside each other and remain distinguishable in the report.
• Shared HTTP provider base (#48): the four providers now derive from HttpLlmHealingProvider, which owns orchestration, validation and retry wiring. Behaviour-neutral; public constructors are unchanged.

Known Limitations

• Calibration: The thresholds shipped in this release are initial estimates, not values derived from a real-world benchmark dataset: MinimumEvidenceWeight (0.40), MinimumCandidateMargin (0.05), MinimumSemanticScore (0.01), MinimumConsensusVotes (2), the 10s Retry-After ceiling, and default AssertGenerationMode (Strict). Each is documented with its rationale and is subject to recalibration under #15.
• Accuracy: Healing accuracy is 100% on the bundled WinForms and WPF demo case studies. The false-positive rate on organic, third-party applications has not been measured (#15).
• LLM Providers: Provider integrations are covered by mocked-HTTP unit tests, and the Windows CI leg additionally makes a real Gemini call on every run when GEMINI_API_KEY is configured (a non-gating comparison harness). What is not covered is a gating live assertion across providers, so consensus itself has never been exercised against real models; llm-smoke.yml runs one OpenAI-compatible endpoint on manual dispatch only, after GitHub Models began returning HTTP 410 ahead of its retirement (#21, #44). Nightly multi-provider collection is tracked in #47.
• Platform: Desktop automation is Windows-only (FlaUI / UIA3). The core heuristic engine and web automation path are cross-platform and verified on Linux in CI (#13).

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery
• AutomationSandbox.IntentAutomation
• AutomationSandbox.PlaywrightLiveExploration

This release publishes package files as GitHub Release assets only. It does not push to nuget.org or GitHub Packages.

v0.2.0-beta.1

Automation Sandbox v0.2.0-beta.1

Preview NuGet package artifact release.

Highlights

• Locator exception gating (#2): Healing retries are strictly gated to locator-resolution failures rather than blindly retrying arbitrary exceptions.
• Evidence coverage gate (#3): Signals with missing metadata on both sides (empty names, null parents, zero bounds) are excluded from the denominator so missing data neither penalizes nor produces a false-perfect 1.00 score.
• Runner-up margin gate (#4): Top candidates with ambiguous margins over the runner-up (best - runnerUp < 0.05) are classified as ambiguous and flagged for manual review instead of falsely marked confident.
• Intent semantic overlap gate (#5): Desktop and web exploration bridges enforce a minimum semantic score (MinimumSemanticScore = 0.01) and candidate margin, routing semantically unrelated matches to review without persisting them.
• LLM candidate explainability and divergence tracking (#6): LLM healing results carry the selected candidate's own score and score breakdown, with explicit divergence tracking when the LLM overrides the heuristic winner.
• Structured assertion generation (#9): Test generators emit structured assertions (AssertionKind, ExpectedValue) with configurable Strict and Lenient modes, replacing false-green presence checks with real assertions.
• Positioning, Linux CI, and packaging parity (#12, #13, #14): Honest scope positioning in documentation, dual-OS CI matrix across Windows (net48) and Linux (net8.0), and complete packaging parity across all seven libraries.

Known Limitations

• Calibration: The thresholds shipped in this release are initial estimates, not values derived from a real-world benchmark dataset: MinimumEvidenceWeight (0.40), MinimumCandidateMargin (0.05), MinimumSemanticScore (0.01), and default AssertGenerationMode (Strict). Each is documented with its rationale and is subject to recalibration under #15.
• Accuracy: Healing accuracy is 100% on the bundled WinForms and WPF demo case studies. The false-positive rate on organic, third-party applications has not been measured (#15).
• LLM Providers: Provider integrations are covered by mocked-HTTP unit tests, and the Windows CI leg additionally makes a real Gemini call on every run when GEMINI_API_KEY is configured (LlmHealingEvaluationTests, a non-gating comparison harness). What is not covered is a gating live assertion across providers: llm-smoke.yml exercises one OpenAI-compatible endpoint on manual dispatch only, after GitHub Models began returning HTTP 410 ahead of its retirement (#21, #44).
• Platform: Desktop automation is Windows-only (FlaUI / UIA3). The core heuristic engine and web automation path are cross-platform and verified on Linux in CI (#13).

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery
• AutomationSandbox.IntentAutomation
• AutomationSandbox.PlaywrightLiveExploration

This release publishes package files as GitHub Release assets only. It does not push to nuget.org or GitHub Packages.

v0.1.0-preview.2

Automation Sandbox v0.1.0-preview.2

Preview NuGet package artifact release.

Highlights

• Intent automation pipeline: plan, match, record, and generate tests from a single API.
• Playwright C# and TypeScript test skeleton generation from recorded intent locators.
• Intent flow JSON/HTML reports for reviewable candidate and locator decisions.

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery
• AutomationSandbox.IntentAutomation

This release publishes package files as GitHub Release assets only. It does not push to nuget.org or GitHub Packages.

v0.1.0-preview.1

Automation Sandbox v0.1.0-preview.1

Preview NuGet package artifact release.

Packages

• AutomationSandbox.UiModel
• AutomationSandbox.SelfHealing
• AutomationSandbox.LlmHealing
• AutomationSandbox.Discovery
• AutomationSandbox.WebDiscovery

This release publishes package files as GitHub Release assets only. It does not push to nuget.org or GitHub Packages.