AlloyDB Cross-Project Connectivity via nginx TCP Proxy

Validated Learning Experiment: Successfully proven nginx TCP proxy approach for cross-project AlloyDB connectivity, bypassing GCP's transitive VPC peering limitation.

---

📊 Experiment Results

✅ VALIDATED - PRODUCTION READY

This repository contains a complete validated learning experiment demonstrating cross-project AlloyDB connectivity using an nginx TCP proxy VM. The approach has been tested and validated with:

• ✅ 100% success rate across all test scenarios
• ✅ 40ms average database connection time
• ✅ ~6ms read query latency
• ✅ ~10ms write query latency
• ✅ Zero downtime implementation
• ✅ Cost-effective (~$10/month for proxy infrastructure)

📄 Full Results: See EXPERIMENT_RESULTS.md for comprehensive validation data and findings.

---

🎯 Problem Statement

The Challenge: Transitive VPC Peering Limitation

When AlloyDB is deployed with Private Service Access (PSA) in Project A, and you need to connect from Cloud Run in Project B, direct VPC peering fails due to GCP's transitive peering limitation.

❌ What Doesn't Work:

Cloud Run (Project B)
  → VPC Connector
  → Project B VPC
  → [VPC Peering]
  → Project A VPC
  → [❌ Cannot traverse]
  → Google-managed VPC (AlloyDB PSA)

Traffic cannot traverse multiple VPC peering connections.

✅ The Solution: nginx TCP Proxy

Place an nginx proxy VM in Project A's VPC that has access to AlloyDB via PSA:

Cloud Run (Project B)
  → VPC Connector (10.8.0.0/28)
  → Project B VPC (10.2.0.0/20)
  → [VPC Peering - ACTIVE]
  → Project A VPC (10.1.0.0/20)
  → nginx Proxy VM (10.1.0.2:5432) ✅
  → AlloyDB (10.170.0.2:5432) ✅

Why It Works: The proxy VM terminates the connection in Project A, where it has direct PSA access to AlloyDB.

---

🏗️ Architecture

High-Level Diagram

┌──────────────────────────────────┬──────────────────────────────────┐
│         PROJECT B                │         PROJECT A                │
│                                  │                                  │
│  ┌──────────────┐                │  ┌──────────────┐               │
│  │  Cloud Run   │                │  │nginx Proxy VM│               │
│  │  Flask App   │                │  │Port 5432     │               │
│  └──────┬───────┘                │  └──────┬───────┘               │
│         │                         │         │                       │
│         ▼                         │         ▼                       │
│  ┌──────────────┐                │  ┌──────────────┐               │
│  │VPC Connector │                │  │   AlloyDB    │               │
│  │10.8.0.0/28   │                │  │10.170.0.2    │               │
│  └──────┬───────┘                │  └──────────────┘               │
│         │                         │         ▲                       │
│         ▼                         │         │                       │
│  ┌──────────────┐                │         │                       │
│  │  Project B   │    VPC Peering │   Private Service               │
│  │     VPC      │◄───────────────┼─► Access (PSA)                  │
│  │10.2.0.0/20   │                │                                  │
│  └──────────────┘                │                                  │
└──────────────────────────────────┴──────────────────────────────────┘

Key Components

Project A (Database):

• AlloyDB cluster (PostgreSQL 16.9)
• nginx TCP proxy VM (e2-micro)
• VPC with Private Service Access
• Firewall rules for cross-project access

Project B (Application):

• Cloud Run service with Flask test app
• Serverless VPC Connector
• VPC peered to Project A

Phase 3 (Connectivity):

• Bidirectional VPC peering
• Firewall rules for both VPC and Connector CIDRs
• Custom route export/import

---

📚 Documentation

Document	Description
EXPERIMENT_RESULTS.md	Complete validation results, performance metrics, findings
USAGE.md	API reference, testing guide, load testing instructions
PROXY_SETUP_GUIDE.md	Quick guide to add proxy to existing infrastructure
ADR-001	Architecture decision record explaining the approach

---

🚀 Quick Start

Prerequisites

• Two GCP projects (Project A for AlloyDB, Project B for Cloud Run)
• Terraform >= 1.0
• gcloud CLI
• Docker

Deployment

1. Set up Project A (AlloyDB + Proxy)

cd terraform/project-a
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your project ID

terraform init
terraform apply

# Save outputs
terraform output

2. Build and push Flask app

cd ../../app
PROJECT_B_ID="your-project-b-id"

# Create Artifact Registry
gcloud artifacts repositories create alloydb-test \
  --repository-format=docker \
  --location=us-central1 \
  --project=$PROJECT_B_ID

# Build and push
gcloud auth configure-docker us-central1-docker.pkg.dev
docker build --platform linux/amd64 \
  -t us-central1-docker.pkg.dev/$PROJECT_B_ID/alloydb-test/test-app:latest .
docker push us-central1-docker.pkg.dev/$PROJECT_B_ID/alloydb-test/test-app:latest

3. Set up Project B (Cloud Run)

cd ../terraform/project-b
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with Project B ID and proxy IP from step 1

terraform init
terraform apply

4. Set up VPC Peering

cd ../phase-3-peering
cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with both project IDs

terraform init
terraform apply

5. Test the connection

# Get Cloud Run URL
cd ../project-b
SERVICE_URL=$(terraform output -raw cloud_run_url)

# Test health
curl $SERVICE_URL/health

# Test database connectivity
curl $SERVICE_URL/test

Expected response:

{
  "status": "success",
  "connection_time_ms": 40.27,
  "read_test": {
    "status": "passed",
    "query_time_ms": 5.24
  },
  "write_test": {
    "status": "passed",
    "query_time_ms": 8.62
  }
}

---

📦 Repository Structure

alloydb-connection-experiment/
├── README.md                          # This file
├── EXPERIMENT_RESULTS.md              # Validation results and findings
├── USAGE.md                           # API reference and usage guide
├── PROXY_SETUP_GUIDE.md               # Quick setup for existing infrastructure
│
├── docs/
│   └── adr/
│       └── 0001-cross-project-alloydb-connectivity.md
│
├── terraform/
│   ├── project-a/                     # AlloyDB + nginx proxy
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   ├── vpc.tf
│   │   ├── alloydb.tf
│   │   ├── proxy-vm.tf
│   │   ├── firewall.tf
│   │   ├── scripts/nginx-setup.sh
│   │   └── terraform.tfvars.example
│   │
│   ├── project-b/                     # Cloud Run + VPC Connector
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   ├── vpc.tf
│   │   ├── cloud-run.tf
│   │   └── terraform.tfvars.example
│   │
│   └── phase-3-peering/               # VPC peering configuration
│       ├── main.tf
│       ├── variables.tf
│       ├── outputs.tf
│       ├── peering.tf
│       └── terraform.tfvars.example
│
├── app/
│   ├── Dockerfile
│   ├── requirements.txt
│   └── test_connection.py             # Flask app with test endpoints
│
├── load-test/
│   ├── README.md                      # Load testing documentation
│   ├── load_test.py                   # Python load testing tool
│   ├── simple_load_test.sh            # Bash load testing (no dependencies)
│   ├── run_load_tests.sh              # Comprehensive test suite
│   └── requirements.txt
│
└── scripts/
    ├── setup.sh                        # Automated setup
    ├── test.sh                         # Run connectivity tests
    └── teardown.sh                     # Clean up resources

---

🧪 Features

API Endpoints

The Flask test application provides:

• /health - Health check (no database required)
• /test - Comprehensive connectivity test (read + write)
• /query - Execute custom SELECT queries with statistics
• /config - Display configuration (debugging)

See USAGE.md for complete API documentation.

Load Testing

Multiple load testing options:

# Simple bash test (no dependencies)
cd load-test
./simple_load_test.sh https://your-cloud-run-url 10

# Python load test (advanced metrics)
python3 load_test.py --url https://your-cloud-run-url \
  --test connection --requests 50 --concurrency 10

# Comprehensive test suite
./run_load_tests.sh

See load-test/README.md for details.

---

📊 Performance Results

Based on validated experiment (October 2025):

Metric	Value	Notes
Success Rate	100%	Zero failures across all tests
Connection Time	40ms (avg)	Cloud Run → AlloyDB via proxy
Read Query	6ms (avg)	Simple SELECT operations
Write Query	10ms (avg)	INSERT operations
Total Latency	<100ms	Including HTTP overhead
Throughput	10+ req/sec	With concurrency of 10

Infrastructure Cost: ~$10/month (e2-micro proxy + VPC Connector)

---

🔍 Key Findings

What Worked

✅ nginx TCP proxy adds minimal latency (~2-5ms) ✅ VPC peering established reliably ✅ 100% success rate under concurrent load ✅ Simple to monitor (nginx access/error logs) ✅ Cost-effective solution

Lessons Learned

⚠️ VPC Connector CIDR must be explicitly allowed in firewall rules (not just main VPC CIDR) ⚠️ nginx stream module requires explicit installation and loading ⚠️ Cloud Run container must start successfully even if database is unreachable ⚠️ AlloyDB has minimum configuration values (e.g., max_connections >= 200)

See EXPERIMENT_RESULTS.md for complete findings.

---

🛠️ Troubleshooting

Quick Checks

# 1. Verify VPC peering is ACTIVE
gcloud compute networks peerings list \
  --network=project-a-vpc \
  --project=YOUR-PROJECT-A-ID

# 2. Check nginx is listening on port 5432
gcloud compute ssh alloydb-proxy \
  --project=YOUR-PROJECT-A-ID \
  --zone=us-central1-a \
  --tunnel-through-iap \
  --command="ss -tlnp | grep 5432"

# 3. Test from Cloud Run
curl https://your-cloud-run-url/test

See USAGE.md for comprehensive troubleshooting guide.

---

🧹 Cleanup

# Destroy in reverse order
cd terraform/phase-3-peering && terraform destroy
cd ../project-b && terraform destroy
cd ../project-a && terraform destroy

Or use the automated script:

./scripts/teardown.sh

---

📖 Use Cases

✅ Good Fit For

• AlloyDB with Private Service Access (cannot migrate to PSC)
• Cross-project connectivity requirements
• Low-to-medium connection volume (<1000 concurrent)
• Latency requirements <100ms acceptable
• Cost-sensitive projects avoiding PSC migration downtime

❌ Not Recommended For

• Very high throughput (>10,000 concurrent connections)
• Ultra-low latency requirements (<10ms total)
• When PSC migration is feasible
• Compliance requires no intermediary components

---

🤝 Contributing

This is a validated experiment repository. If you find issues or have improvements:

1. Open an issue describing the problem
2. Reference the specific component (terraform, app, docs)
3. Include environment details and error messages

---

📄 License

This experiment code is provided as-is for learning and validation purposes.

---

🙏 Acknowledgments

• GCP Documentation: VPC Peering, AlloyDB, Cloud Run
• nginx: Stream module for TCP proxying
• Lean Startup Methodology: Validated learning approach

---

📞 Support

For questions about this experiment:

1. Review EXPERIMENT_RESULTS.md for validation details
2. Check USAGE.md for usage and troubleshooting
3. See PROXY_SETUP_GUIDE.md for retrofitting existing setups
4. Review ADR in docs/adr/0001-cross-project-alloydb-connectivity.md

---

Experiment Status: ✅ VALIDATED - PRODUCTION READY Date: October 27-28, 2025 Recommendation: Proceed with production implementation

🤖 Generated with Claude Code