Skip to content

hotfix(security): drop global Database Query plugin block#1339

Merged
OleksandrUA merged 1 commit into
stagingfrom
SEC-drop-db-integration-block-2026-05-21
May 21, 2026
Merged

hotfix(security): drop global Database Query plugin block#1339
OleksandrUA merged 1 commit into
stagingfrom
SEC-drop-db-integration-block-2026-05-21

Conversation

@OleksandrUA
Copy link
Copy Markdown

Summary

Drops the block_database_integration_type trigger installed during the 2026-05-21 incident (migration 0082). The trigger refused every INSERT/UPDATE on integrations where type='database', disabling the DB Query plugin for everyone while the active attack was being contained.

That block is no longer the right gate. Other defences remain in place:

  • block_executions trigger on workflow_executions (migration 0082) rejects inserts when the workflow owner is deactivated_at IS NOT NULL or the workflow is deleted_at IS NOT NULL
  • block_user_signup trigger on users allows only @techops.services / @keeperhub.com signups
  • SSRF guard parity across HTTP and database plugin SQL paths (KEEP-603)
  • NetworkPolicy on workflow-runner pods blocking egress to 169.254.0.0/16, fe80::/10, fc00::/7
  • IMDSv2 hop-limit=1 on Karpenter NodeClasses
  • Attacker user accounts hard-deleted, all their channels (sessions, OAuth, api_keys, org_api_keys, integrations) wiped

Legitimate operators (Sky engineers monitoring chief-keeper-spells, Neon analytics DBs etc.) need the DB Query plugin back to re-create their integrations after the incident-wide credential rotation.

State on prod

The trigger + function were already dropped on the prod DB earlier today via direct DDL during incident response. This migration is idempotent via IF EXISTS, so the next pnpm db:migrate on prod is a no-op for the trigger drop but advances the journal to record the change.

Test plan

  • Migration SQL is idempotent (DROP TRIGGER IF EXISTS, DROP FUNCTION IF EXISTS)
  • No schema column changes (snapshot copied from 0082)
  • CI green
  • On staging merge: confirm pnpm db:migrate advances the journal cleanly
  • On prod merge: confirm pnpm db:migrate advances the journal cleanly; the trigger remains absent (already dropped)
  • Dumitru can save the Neon DB integration after this lands

Migration 0082 installed block_database_integration_type on integrations,
refusing every INSERT/UPDATE that set type='database' so the DB Query plugin
was disabled for everyone during the active incident.

That block is no longer the right gate. The other defences from incident
response are sufficient and stay in place:

- block_executions trigger on workflow_executions rejects inserts when the
  workflow owner is deactivated_at IS NOT NULL or the workflow itself is
  deleted_at IS NOT NULL (migration 0082)
- SSRF guard parity across HTTP and database plugin SQL paths (KEEP-603)
- NetworkPolicy on workflow-runner pods blocks egress to 169.254.0.0/16,
  fe80::/10, fc00::/7
- IMDSv2 hop-limit=1 on Karpenter NodeClasses
- Compromised user accounts hard-deleted, their channels (sessions, OAuth,
  api_keys, org_api_keys, integrations) wiped

Legitimate operators (Sky engineers monitoring chief-keeper-spells, Neon
analytics DBs etc.) need the DB Query plugin back to re-create their
integrations after the incident-wide credential rotation. The trigger is
already dropped on prod via direct DB action during incident response; this
migration brings the migration history in line so a fresh DB built from
migrations matches prod.

Idempotent via IF EXISTS.
@OleksandrUA OleksandrUA merged commit 75eb284 into staging May 21, 2026
31 checks passed
@OleksandrUA OleksandrUA deleted the SEC-drop-db-integration-block-2026-05-21 branch May 21, 2026 17:11
@github-actions
Copy link
Copy Markdown

🧹 PR Environment Cleaned Up

The PR environment has been successfully deleted.

Deleted Resources:

  • Namespace: pr-1339
  • All Helm releases (Keeperhub, Scheduler, Event services)
  • PostgreSQL Database (including data)
  • LocalStack, Redis
  • All associated secrets and configs

All resources have been cleaned up and will no longer incur costs.

@github-actions
Copy link
Copy Markdown

ℹ️ No PR Environment to Clean Up

No PR environment was found for this PR. This is expected if:

  • The PR never had the deploy-pr-environment label
  • The environment was already cleaned up
  • The deployment never completed successfully

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant