Skip to content

chore: stabilize sdk compliance adapter#617

Merged
marandaneto merged 2 commits into
mainfrom
chore/sdk-compliance-feature-flags
May 22, 2026
Merged

chore: stabilize sdk compliance adapter#617
marandaneto merged 2 commits into
mainfrom
chore/sdk-compliance-feature-flags

Conversation

@marandaneto
Copy link
Copy Markdown
Member

@marandaneto marandaneto commented May 22, 2026

💡 Motivation and Context

Fix the SDK compliance adapter state handling for the Feature_Flags compliance suite.

The adapter could deadlock during test reset because it held SDKState.lock while client.shutdown() flushed pending events through the patched transport, which also records requests under the same lock. Feature flag side-effect events could also be flushed after an adapter action returned, leaking into the next compliance test's mock-server state.

This also updates the compliance workflow to use the released posthog-sdk-test-harness fix from PostHog/posthog-sdk-test-harness#19. The reusable workflow is pinned to full commit SHA 85e2901ea3260a28e07a83086d59b4fb4dfc814f, and the harness image is pinned by digest.

💚 How did you test it?

  • uv run --with flask --with python-dateutil python sdk_compliance_adapter/adapter.py
  • From /Users/marandaneto/Github/posthog-sdk-test-harness:
    • uv run posthog-test-harness run --adapter-url http://localhost:18080 --suite feature_flags --output text --concurrency 1
    • Result: 16/16 Feature_Flags tests passed
  • uv run --with pyyaml python - <<'PY' ... yaml.safe_load(...) ... PY
  • docker manifest inspect 'ghcr.io/posthog/sdk-test-harness:main-85e2901@sha256:4c8eac34e7ff66554a2c6947788c0a42b82456bc949c03bd8f6b9a10bef23ef5'

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file

@marandaneto marandaneto requested a review from a team as a code owner May 22, 2026 08:49
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 22, 2026

Reviews (1): Last reviewed commit: "chore: stabilize sdk compliance adapter" | Re-trigger Greptile

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 22, 2026

posthog-python Compliance Report

Date: 2026-05-22 12:07:28 UTC
Duration: 175945ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 512ms
Format Validation.Event Has Uuid 1506ms
Format Validation.Event Has Lib Properties 1506ms
Format Validation.Distinct Id Is String 1505ms
Format Validation.Token Is Present 1506ms
Format Validation.Custom Properties Preserved 1506ms
Format Validation.Event Has Timestamp 1505ms
Retry Behavior.Retries On 503 9517ms
Retry Behavior.Does Not Retry On 400 3504ms
Retry Behavior.Does Not Retry On 401 3506ms
Retry Behavior.Respects Retry After Header 9512ms
Retry Behavior.Implements Backoff 23526ms
Retry Behavior.Retries On 500 7503ms
Retry Behavior.Retries On 502 7507ms
Retry Behavior.Retries On 504 7510ms
Retry Behavior.Max Retries Respected 23527ms
Deduplication.Generates Unique Uuids 1495ms
Deduplication.Preserves Uuid On Retry 7514ms
Deduplication.Preserves Uuid And Timestamp On Retry 14510ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7514ms
Deduplication.No Duplicate Events In Batch 1502ms
Deduplication.Different Events Have Different Uuids 1506ms
Compression.Sends Gzip When Enabled 1505ms
Batch Format.Uses Proper Batch Structure 1505ms
Batch Format.Flush With No Events Sends Nothing 1004ms
Batch Format.Multiple Events Batched Together 1505ms
Error Handling.Does Not Retry On 403 3508ms
Error Handling.Does Not Retry On 413 3506ms
Error Handling.Retries On 408 7512ms

Feature_Flags Tests

16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 1002ms
Request Payload.Flags Request Uses V2 Query Param 1005ms
Request Payload.Flags Request Hits Flags Path Not Decide 1006ms
Request Payload.Flags Request Omits Authorization Header 1005ms
Request Payload.Token In Flags Body Matches Init 1005ms
Request Payload.Groups Round Trip 1005ms
Request Payload.Groups Default To Empty Object 1006ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It 1005ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 1005ms
Request Payload.Disable Geoip Omitted Defaults To False 1005ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 1006ms
Request Lifecycle.No Flags Request On Init Alone 502ms
Request Lifecycle.No Flags Request On Normal Capture 1506ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 1008ms
Request Lifecycle.Mock Response Value Is Returned To Caller 1002ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 1508ms

@marandaneto marandaneto marked this pull request as draft May 22, 2026 12:01
@marandaneto marandaneto force-pushed the chore/sdk-compliance-feature-flags branch from bc3f7bd to b83bd24 Compare May 22, 2026 12:03
@marandaneto marandaneto force-pushed the chore/sdk-compliance-feature-flags branch from b83bd24 to 131b925 Compare May 22, 2026 12:03
@marandaneto marandaneto marked this pull request as ready for review May 22, 2026 12:17
@marandaneto
Copy link
Copy Markdown
Member Author

ok CI is happy now

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 22, 2026

Reviews (2): Last reviewed commit: "ci: pin sdk compliance harness sha" | Re-trigger Greptile

Copy link
Copy Markdown

@ioannisj ioannisj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very low context approve but changes seem reasonable to me.

@marandaneto marandaneto merged commit 15df3a4 into main May 22, 2026
27 checks passed
@marandaneto marandaneto deleted the chore/sdk-compliance-feature-flags branch May 22, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants