Skip to content

Add polling options, service endpoint, CBOR error handling, and SDK updates#168

Merged
elantiguamsft merged 12 commits intomainfrom
users/jstatia/mst-polling-options
Mar 13, 2026
Merged

Add polling options, service endpoint, CBOR error handling, and SDK updates#168
elantiguamsft merged 12 commits intomainfrom
users/jstatia/mst-polling-options

Conversation

@JeromySt
Copy link
Member

@JeromySt JeromySt commented Mar 7, 2026

Performance: Fast retry policy, tunable polling, structured error handling, and SDK updates

Motivation

The Azure Code Transparency service returns 503 TransactionNotCached when a newly registered entry hasn't propagated to the read replicas yet — typically resolved in well under 1 second. However, the server's Retry-After: 1 header causes the SDK's default retry logic to wait a full second between attempts, adding unnecessary latency to the hot path of entry registration and receipt retrieval. This PR eliminates that bottleneck and adds several supporting features.


🚀 MstTransactionNotCachedPolicy — Fast 503 retry pipeline policy (new)

A custom HttpPipelinePolicy that intercepts 503 TransactionNotCached responses and performs aggressive, in-pipeline retries before the SDK's standard retry logic kicks in.

Parameter Default Effect
retryDelay 250ms Fixed interval between retry attempts
maxRetries 8 Maximum retry attempts (total window ≈ 2 seconds)

How it works:

  1. Intercepts responses matching all of: HTTP 503 + GET method + URI path contains /entries/ + CBOR body with TransactionNotCached error code
  2. Re-sends the request at 250ms intervals until success or max retries exhausted
  3. Non-matching requests (POST, different paths, different error codes) pass through untouched to the SDK's normal retry infrastructure

Key design decisions:

  • Positioned at HttpPipelinePosition.PerRetry — runs inside the SDK's retry loop, minimizing total latency
  • Parses response body as RFC 9290 CBOR problem details to verify the error code (case-insensitive)
  • Buffers non-seekable response streams to ensure the body remains readable across retries
  • Parse failures (invalid CBOR, stream I/O errors) gracefully fall through to SDK retry — no false positives

Configuration via extension method:

options.ConfigureTransactionNotCachedRetry();                         // 250ms, 8 retries (default)
options.ConfigureTransactionNotCachedRetry(TimeSpan.FromMilliseconds(100)); // faster
options.ConfigureTransactionNotCachedRetry(maxRetries: 16);           // longer window

⏱️ MstPollingOptions — Tunable receipt polling (new)

Controls how MakeTransparentAsync polls for the long-running operation receipt.

Property Type Behavior
PollingInterval TimeSpan? Fixed interval between polls (e.g., 250ms)
DelayStrategy DelayStrategy? Custom back-off (exponential, jitter, etc.) — takes precedence
  • Both null → SDK default exponential back-off
  • PollingInterval set → WaitForCompletionAsync(interval, ct)
  • DelayStrategy set → Operation<T>.DelayStrategy assignment (overrides PollingInterval)
  • RegisterCommand and VerifyCommand now default to 250ms fixed polling

📋 CborProblemDetails — RFC 9290 Concise Problem Details parser (new)

Parses CBOR-encoded error responses per RFC 9290:

CBOR Key Field Description
-1 Type URI reference identifying the problem type
-2 Title Human-readable summary
-3 Status HTTP status code
-4 Detail Human-readable explanation
-5 Instance URI reference for this specific occurrence
other Extensions Any additional keys captured as a dictionary

Also accepts string keys ("type", "title", etc.) for interoperability. Handles HalfPrecisionFloat CBOR values consistent with CwtClaims.cs.


🔥 MstServiceException — Structured error wrapping (new)

Wraps Azure.RequestFailedException with parsed CBOR problem details:

catch (RequestFailedException rfEx)
{
    throw MstServiceException.FromRequestFailedException(rfEx);
    // rfEx.ProblemDetails.Title → "TransactionNotCached"
    // rfEx.ProblemDetails.Detail → "Transaction ... is not yet cached"
    // rfEx.StatusCode → 503
}
  • Factory method inspects Content-Type for cbor, extracts and parses the response body
  • Falls back to generic wrapping if CBOR parsing fails
  • ToString() includes all parsed fields and extensions for diagnostics

🔌 TransparencyService.ServiceEndpoint — Base class URI property (new)

  • Auto-derived from CodeTransparencyClient via reflection (best-effort)
  • Explicit URI override available via constructor parameter
  • Available to all TransparencyService implementations

📦 Azure SDK package updates

Package Before After
Azure.Core 1.50.0 1.51.1
Azure.Identity 1.17.0 1.18.0
Azure.Security.CodeTransparency 1.0.0-beta.5 1.0.0-beta.8

🧪 Azure.Core.TestCommon — Test infrastructure (new)

Vendored from azure-sdk-for-net test infrastructure (Azure.Core 1.51.1 era):

  • MockResponse, MockTransport, MockRequest — controlled HTTP responses for testing Azure SDK clients
  • AsyncGate, AsyncValidatingStream, DictionaryHeaders — supporting utilities
  • All classes marked [ExcludeFromCodeCoverage]

🗑️ Removed

  • CtsCommandBase (332 lines) and CtsCommandBaseLoggingTests (482 lines) — superseded by the new polling and retry infrastructure

Test coverage

Area Tests Coverage
MstTransactionNotCachedPolicy 23 Constructor validation, pass-through for non-matching requests, retry-until-success, retry exhaustion, case-insensitive matching, sync/async paths
MstPollingOptions 22 Property defaults, constructor wiring, all MstTransparencyService overloads
CborProblemDetails 41 Null/empty/invalid input, integer keys, string keys, extensions, partial fields, HalfPrecisionFloat
MstServiceException covered in CborProblemDetails tests FromRequestFailedException, BuildErrorMessage, ToString, StatusCode
MstTransparencyService +1 new MakeTransparentAsync wraps RequestFailedException in MstServiceException
Total new 87+

Summary for reviewers

The core performance improvement is the MstTransactionNotCachedPolicy — by retrying at 250ms instead of the server's 1-second Retry-After, entry receipt retrieval that previously took 2–4 seconds now typically completes in under 1 second. Combined with 250ms LRO polling intervals (replacing SDK default exponential back-off), end-to-end registration latency is significantly reduced.

All new code has structured CBOR error handling, cancellation token support, and comprehensive test coverage.

…, and SDK updates

MstPollingOptions  tunable receipt polling:
- PollingInterval (TimeSpan?) for fixed-interval polling
- DelayStrategy (Azure.Core.DelayStrategy?) for custom back-off
- DelayStrategy takes precedence if both are set; null = SDK default

TransparencyService.ServiceEndpoint  base class URI property:
- Auto-derived from CodeTransparencyClient via reflection
- Explicit URI override via constructor parameter

RFC 9290 CBOR problem details parsing:
- CborProblemDetails: parses integer keys (-1..-5) + string keys + extensions
- MstServiceException: FromRequestFailedException() parses CBOR error bodies
- MakeTransparentCoreAsync catches RequestFailedException with structured errors

Azure.Core.TestCommon  test infrastructure:
- MockResponse, MockTransport, MockRequest, etc. ported from azure-sdk-for-net
- Enables testing Azure SDK clients with controlled responses

Azure SDK package updates:
- Azure.Core 1.50.0 -> 1.51.1
- Azure.Identity 1.17.0 -> 1.18.0
- Azure.Security.CodeTransparency 1.0.0-beta.5 -> 1.0.0-beta.8

Tests: 92 pass, 0 fail (51 new). Coverage on new code:
  CborProblemDetails.cs   94.9%
  MstPollingOptions.cs    100%
  MstServiceException.cs  96.9%

This comment was marked as resolved.

actions-user and others added 4 commits March 7, 2026 05:09
- Seal MockRequest and MockResponse to prevent virtual call issues in ctors
- Replace generic catch clauses with specific CborContentException,
  InvalidOperationException, TargetInvocationException in production code
- Use ternary expressions in DictionaryHeaders for cleaner assignment
- Remove unused variable in CborProblemDetailsTests
- Add System.Formats.Cbor using to MstServiceException
- All 92 MST tests pass

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix naming: rename _serviceEndpoint to ServiceEndpointUri (PascalCase)
- Add HalfPrecisionFloat handling in CborProblemDetails.ReadAnyValue
- Fix test: assert on ex.Message and ProblemDetails in BuildErrorMessage test
- Add test: MakeTransparentAsync wraps RequestFailedException in MstServiceException
- Fix comment: update Azure.Core version reference to 1.51.1
- All 93 MST tests pass (1 new)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nt retries

- Custom Azure SDK pipeline policy that handles 503/TransactionNotCached
  with aggressive 250ms retries (up to 8 attempts) instead of server's
  1-second Retry-After header
- CBOR problem details body parsed to verify TransactionNotCached error code
- Policy is scoped: only GET /entries/ 503s are retried; all other requests
  pass through to the SDK's normal retry infrastructure
- Extension method ConfigureTransactionNotCachedRetry() on CodeTransparencyClientOptions
- ToCoseSign1TransparencyService() overload accepting MstPollingOptions
- CodeTransparencyClientHelper now configures the policy by default
- RegisterCommand and VerifyCommand now use 250ms LRO polling interval
- Removed unused CtsCommandBase and CtsCommandBaseLoggingTests
- Updated documentation with new extension methods and retry behavior
Jstatia and others added 2 commits March 11, 2026 13:32
Replace auto-property overrides with explicit backing fields in MockRequest
and MockResponse so constructors assign fields directly instead of invoking
virtual property setters. Both classes are sealed so the original code was
safe at runtime, but CodeQL still flags the pattern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace foreach+if with LINQ .Any() for cleaner extension value filtering
- Replace bare catch with catch (Exception) to satisfy generic-catch-clause alert

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace catch (Exception) with IOException, ObjectDisposedException,
and NotSupportedException to satisfy CodeQL generic-catch-clause alert.
CborProblemDetails.TryParse already handles CBOR exceptions internally,
so only stream I/O failures need catching here.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This comment was marked as resolved.

- Add missing using directives in doc code snippets (Azure.Core, Azure.Core.Pipeline)
- Dispose previous Response before retry in MstTransactionNotCachedPolicy
- Replace single Stream.Read with CopyTo for reliable body buffering
- Move BinaryDataExtensions from System to CoseSign1.Transparent.MST namespace
- Align TryGetMstEntryId docs with actual behavior (returns empty string, not null)
- Update MstPollingOptions DelayStrategy docs to describe WaitForCompletionAsync overload
- Make pollingOptions parameter nullable in MstTransparencyService constructors
- Fix CHANGELOG version tag: v.1.5.5 -> v1.5.5

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Jstatia and others added 2 commits March 11, 2026 14:15
Added using statement to bodyBuffer MemoryStream in
TryGetEntryIdFromBody to ensure proper disposal.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
elantiguamsft
elantiguamsft previously approved these changes Mar 12, 2026
- Link RFC 9290 citations as hyperlinks in CborProblemDetails XML docs
- Add descriptive comments explaining Response disposal before retry
- Pass CancellationToken to Task.Delay; add ThrowIfCancellationRequested for sync path
- Extract DefaultPollingIntervalMs constant in MstCommandBase (used by Register/Verify)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@elantiguamsft elantiguamsft merged commit 83174f0 into main Mar 13, 2026
11 checks passed
@elantiguamsft elantiguamsft deleted the users/jstatia/mst-polling-options branch March 13, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants