Skip to content

tsan: investigate and fix TSAN failure for createJsonFormatter StatusOr change#45836

Closed
Copilot wants to merge 1 commit into
mainfrom
copilot/fix-github-actions-tsan-job
Closed

tsan: investigate and fix TSAN failure for createJsonFormatter StatusOr change#45836
Copilot wants to merge 1 commit into
mainfrom
copilot/fix-github-actions-tsan-job

Conversation

Copilot AI commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Context

PR #45831 changes SubstitutionFormatStringUtils::createJsonFormatter return type from FormatterPtr to absl::StatusOr<FormatterPtr>, and the TSAN CI job failed during that run (exit code 3 — test failure, not build failure).

Investigation

The session exhausted quota before a conclusive root cause was identified or a code fix implemented. Key findings from the analysis:

  • CI artifacts were inaccessible: Job 83338832915 logs only showed post-processing tail; the ci-report-tsan.json artifact contained only disk usage stats; the EngFlow BES invocation URL was not reachable externally.
  • Two concurrent changes are in scope: The PR's formatter StatusOr changes and the TLS read_disabled_ logic introduced in main at df2c77d (tls: fix connection close detection while handshake is waiting), which ships a new AsyncCertSelectionCancellationObservedOnDownstreamClose integration test.
  • ASAN passed / TSAN failed: This is characteristic of a data race, not a memory safety issue — pointing away from the formatter change itself.
  • Prime suspect: CancellableSelectionHandle destructor in async_cert_selector.h holds an Event::TimerPtr and CertificateSelectionCallbackPtr; if destroyed off the dispatcher thread (e.g., when the downstream connection closes and tears down SslExtendedSocketInfoImpl), TSAN would flag it as a data race.

Status

⚠️ Incomplete — no code changes were committed. Further work is needed to:

  1. Retrieve the actual TSAN report (stack traces) from a fresh CI run
  2. Confirm the CancellableSelectionHandle destruction threading hypothesis
  3. Implement the fix (likely ensuring cert-selector handle destruction is posted back to the dispatcher thread, or guarding access with appropriate synchronization)

@repokitteh-read-only

Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #45836 was opened by Copilot.

see: more, trace.

Copilot AI changed the title [WIP] Fix failing GitHub Actions job for tsan tsan: investigate and fix TSAN failure for createJsonFormatter StatusOr change Jun 25, 2026
Copilot AI requested a review from yanavlasov June 25, 2026 11:48
@yanavlasov yanavlasov closed this Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants