feat(experimentation): sync warehouse connection environment keys to ingestion redis#7588
Conversation
…ingestion redis Adds AFTER_CREATE / AFTER_DELETE hooks on WarehouseConnection that enqueue tasks to set/delete the environment's API key in the ingestion server's Redis Cluster under the `experimentation:environment_keys:` namespace.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
github-actions
Bot
added
api
Docker builds report
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements a synchronization mechanism between warehouse connections and an event ingestion Redis cluster. It introduces a new service for Redis communication, background tasks for asynchronous processing, and lifecycle hooks on the WarehouseConnection model. Feedback highlights a potential ValueError when the Redis URL is unconfigured and a logic error in the deletion hook where the environment key might be prematurely removed if multiple connections exist for the same environment.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7588 +/- ##
==========================================
+ Coverage 98.49% 98.51% +0.01%
==========================================
Files 1417 1436 +19
Lines 53749 54363 +614
==========================================
+ Hits 52939 53553 +614
Misses 810 810 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Failed testsfirefox › tests/project-permission-test.pw.ts › Project Permission Tests › Project-level permissions control access to features, environments, audit logs, and segments @enterprise
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
|
…atalogue Drop the try/except RedisError in ingestion_sync_service — failures should surface to the task processor rather than being swallowed. Regenerate the events catalogue to include the new ingestion_sync events.
github-actions
Bot
added
docs
Visual Regression19 screenshots compared. See report for details. |
Push the autouse mock down from _get_client to RedisCluster.from_url so the real _get_client body runs in tests. Add a direct test that asserts the socket timeout / keepalive kwargs we pass to RedisCluster.from_url.
github-actions
Bot
added
docs
Point both admin-api and task-processor task definitions at the existing serverless-redis-cache cluster (rediss://...:6379) in staging and production so the experimentation ingestion sync hooks can publish environment keys.
github-actions
Bot
added
docs
github-actions
Bot
added
feature
Zaimwa9
left a comment
Zaimwa9
left a comment
There was a problem hiding this comment.
I let you decide on the log
…rvice
The set/delete success logs added no operational value over what the task
processor already records. Regenerate the events catalogue to drop the
ingestion_sync.environment_key.{set,deleted} entries.
github-actions
Bot
added
feature
2 participants
docs/if required so people know about the feature.Changes
Contributes to experimentation warehouse ingestion sync.
Adds
AFTER_CREATE/AFTER_DELETEhooks onWarehouseConnectionthat enqueue background tasks to publish the environment's API key to the ingestion server's Redis Cluster under theexperimentation:environment_keys:namespace:SET experimentation:environment_keys:<env-api-key> ""on connection createDEL experimentation:environment_keys:<env-api-key>on connection deleteNew pieces:
experimentation/ingestion_sync_service.py— owns the Redis Cluster client (lazy,from_urlwithsocket_timeout=1,socket_keepalive=True) and the key-prefix constant. Logsingestion_sync.environment_key.{set,deleted,failed}via structlog; swallowsRedisErrorso a Redis outage cannot break Django requests.experimentation/tasks.py—add_environment_key_to_ingestion/delete_environment_key_from_ingestiontask handlers.INGESTION_REDIS_URLsetting (empty default).How did you test this code?
tests/unit/experimentation/test_ingestion_sync_service.py— set/delete write the expected key,RedisErroris logged and swallowed.tests/unit/experimentation/test_tasks.py— task handlers delegate to the service.tests/unit/experimentation/test_models.py— hooks enqueue the right task withenvironment.api_key.tests/unit/experimentation/conftest.pymocks_get_clientso no test touches real Redis.pytest tests/unit/experimentation/— 43/43 pass.mypyon the touched files — clean.