Skip to content

feat: AlloyDB dual-write for gmail_messages - #5

Merged
dtomic15 merged 1 commit into
mainfrom
feat/alloydb-dual-write
May 6, 2026
Merged

feat: AlloyDB dual-write for gmail_messages#5
dtomic15 merged 1 commit into
mainfrom
feat/alloydb-dual-write

Conversation

@dtomic15

@dtomic15 dtomic15 commented May 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds optional AlloyDB dual-write alongside existing BigQuery writes
  • When ALLOYDB_URL env var is set, the scraper writes to both BQ and AlloyDB
  • Adds backfill endpoint ({"action": "backfill"}) to copy existing BQ data to AlloyDB
  • All AlloyDB operations are non-fatal — BQ path is completely untouched

Why

The Envision OS MCP tool gmail_search_bigquery was repointed to AlloyDB in Phase 268 (BQ retirement), but the gmail-scraper still only writes to BigQuery. This means emails (including flow@envsn.com) are invisible to Envision OS / Claude users.

Deploy steps (after merge)

  1. Update Cloud Run with VPC access + AlloyDB secret:
    gcloud run services update gmail-scraper \
      --region=us-central1 --project=claude-mcp-457317 \
      --network=default --subnet=default \
      --vpc-egress=private-ranges-only \
      --update-secrets=ALLOYDB_URL=envision-alloydb-url:latest
  2. Trigger backfill to sync existing BQ data:
    curl -X POST <service-url> \
      -H "Content-Type: application/json" \
      -d '{"action": "backfill"}'

Test plan

  • Verify existing BQ writes still work (check Cloud Run logs for BigQuery insert messages)
  • Verify AlloyDB dual-write logs appear after VPC + secret configured
  • Run backfill for flow@envsn.com: {"action": "backfill", "user_email": "flow@envsn.com"}
  • Query via Envision OS: gmail_search_bigquery(user_email="flow@envsn.com")

🤖 Generated with Claude Code

The gmail-scraper writes to BigQuery but the Envision OS MCP tool
reads from AlloyDB (repointed in Phase 268). This adds optional
AlloyDB dual-write so emails are queryable via Envision OS / Claude.

- Add pg8000 dependency for pure-Python PostgreSQL connectivity
- Add AlloyDB connection, table creation, and upsert helpers
- Dual-write to AlloyDB alongside BigQuery (non-fatal, try/except)
- Add backfill endpoint to copy existing BQ data to AlloyDB
- ALLOYDB_URL env var controls opt-in (no change when unset)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@dtomic15
dtomic15 merged commit 08e883a into main May 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant