aws-vector-hosting-comparison

Hands-on cost and latency comparison across four vector stores on AWS, in one Terraform module and a handful of Python scripts.

Companion code to the blog post "The Real Cost of Vector Storage: S3 Vectors vs OpenSearch vs pgvector vs Pinecone" on darryl-ruggles.cloud.

What it does

Loads the same 50,000 Titan Embed V2 (1024-dim) embeddings into four vector stores, runs identical query workloads against each, captures real p50/p95/p99 latency, and prints a modeled monthly cost at 10M, 100M, and 1B scale across six workload shapes.

Store	Where it runs	Pricing model
Amazon S3 Vectors	AWS-native, GA Dec 2025	per-GB storage + per-GB write + per-query + per-TB data-processed
Amazon OpenSearch Serverless NextGen	AWS-native, GA May 28 2026, VECTORSEARCH collection in a collection group	OCU-hour when warm, $0 compute when idle (10-min idle timeout) + storage
Aurora PostgreSQL Serverless v2 + pgvector	AWS-native, engine 17.4	ACU-hour + storage + I/O
Pinecone Serverless	Third-party, AWS us-east-1 (optional)	storage + Read Units + Write Units + $50/mo minimum

Prerequisites

• An AWS account. The Makefile defaults to AWS_PROFILE=default; if your profile is named something else, either export AWS_PROFILE=your-profile-name in your shell or drop AWS_PROFILE=your-profile-name into a .env file at the repo root (the Makefile auto-sources .env).
• Bedrock model access for amazon.titan-embed-text-v2:0 in us-east-1.
• Terraform >= 1.13.
• uv for Python deps. Python 3.14 is installed automatically by uv.
• (Optional) A Pinecone account. The free Starter tier is sufficient for this demo - 2 GB storage / 2M Write Units / 1M Read Units per month, no minimum, no credit card. The 50K-vector demo uses ~250 MB and ~15K Read Units per bench run, well inside the free allowance.

Quick start

# (optional) export AWS_PROFILE=your-profile-name   # or put it in .env
make init              # uv sync (Python 3.14 + deps)
make plan              # see what AWS resources would be created
make apply             # provision infra (~10 minutes)

make embed             # generate 50k Titan V2 embeddings
make load              # load into S3 Vectors, OpenSearch, Aurora pgvector

make bench             # run the latency benchmark across the AWS stores
make cost              # print modeled cost at 10M / 100M / 1B vectors

make destroy           # tear it all down

Including Pinecone (free Starter tier)

# 1. Sign up at https://www.pinecone.io/ and grab an API key (free, no card).
# 2. cp .env.example .env  and fill in PINECONE_API_KEY=...
# 3. Provision the Secrets Manager entry for the API key:
make apply ENABLE_PINECONE=true
# 4. Load and bench Pinecone alongside the AWS stores:
make load-pinecone
uv run python -m vector_demo.bench --include-pinecone --out data/results/bench.json

Expected total AWS spend for one full run (provision -> embed -> load -> bench -> destroy): well under USD $1 at 50,000 vectors. Embedding 50k docs costs ~$0.04. Aurora and OpenSearch Serverless dominate the rest, both prorated by the hour the cluster is alive.

A note on the scheduled benchmark

make apply also creates an EventBridge schedule that fires the benchmarker Lambda every 6 hours, unattended. This is independent of make bench (which runs from your laptop) - the Lambda runs the same query set from inside us-east-1 and emits EMF metrics (QueryLatencyP50/P95/P99 per store) and X-Ray traces, so CloudWatch builds a trend over time. Each automatic run costs pennies (one Lambda invocation, ~10 Bedrock embeddings, ~30 store queries). If you do the full Quick start sequence in one sitting, the schedule fires zero or one times before make destroy and you can ignore it. If you intentionally leave the infrastructure standing, expect ~4 automatic runs per day. To change the cadence, edit the schedule variable default in infrastructure/variables.tf (e.g. rate(1 day)) before make apply.

Repository layout

.
├── README.md                     # this file
├── Makefile                      # top-level workflow targets
├── pyproject.toml                # Python 3.14, uv project
├── .env.example                  # copy to .env to set PINECONE_API_KEY etc.
├── images/
│   └── architecture.png          # diagram referenced by this README
├── infrastructure/
│   ├── main.tf, variables.tf, outputs.tf, providers.tf, versions.tf
│   └── modules/
│       ├── s3vectors/            # vector bucket + index
│       ├── opensearch/           # NextGen AOSS collection group (via boto3) + collection + policies
│       ├── aurora-pgvector/      # Serverless v2 + Data API
│       └── benchmarker/          # arm64 Lambda + EventBridge
├── calculator/
│   └── src/vector_cost/
│       ├── prices.py             # single source of truth for $$$
│       ├── model.py              # one function per store
│       └── cli.py                # `vector-cost report ...`
├── demo/
│   └── src/vector_demo/
│       ├── corpus.py             # synthetic storm-event docs
│       ├── embed.py              # Titan V2 -> embeddings.npz
│       ├── load_*.py             # one loader per store
│       └── bench.py              # cross-store latency capture
└── lambdas/
    └── benchmarker/
        ├── requirements.txt      # powertools, opensearch-py, ...
        └── src/benchmarker/handler.py

Running the cost calculator without provisioning anything

The calculator is a pure-Python module - no AWS calls. Useful for plugging your real workload numbers in before you decide which store to commit to.

uv run vector-cost one \
    --vectors 100000000 --qpm 250000 --writes 50000 --metadata-kb 0.5

uv run vector-cost report --output markdown   # the canonical 6-scenario table
uv run vector-cost prices                     # current price table as JSON

Security model

• All AWS resources are tagged Project=vector-hosting-comparison.
• IAM roles are least-privilege:
  • The benchmarker Lambda only has s3vectors:QueryVectors / GetVectors / ListVectors (no write).
  • It has aoss:APIAccessAll constrained by aws:RequestedRegion.
  • It has only the Data API verbs on the one Aurora cluster ARN.
  • Optional Pinecone API key lives in Secrets Manager; the role grants secretsmanager:GetSecretValue on that specific ARN only.
• S3 Vectors bucket uses SSE-S3 by default; switch to SSE-KMS in infrastructure/modules/s3vectors/main.tf if you need BYOK.
• Aurora storage is encrypted, IAM database auth is on, and the master password is managed by Secrets Manager (manage_master_user_password = true).
• OpenSearch Serverless is NextGen (post-May 2026 architecture): a VECTORSEARCH collection inside a NextGen collection group, which scales search OCUs to zero on a 10-minute idle timeout. NextGen collection groups require standby_replicas = "ENABLED" and the generation = "NEXTGEN" API flag. The Terraform AWS provider doesn't yet expose the generation field (verified against v6.47), so the module drives boto3 via a null_resource + local-exec (see modules/opensearch/manage_collection_group.py). AWS CLI v2.34.56+ accepts the field too; the boto3 path is used here so the dependency is pinned in pyproject.toml/uv.lock rather than relying on whatever CLI version is on the developer's machine. When the Terraform provider catches up, the null_resource can be swapped for aws_opensearchserverless_collection_group with generation = "NEXTGEN" and the helper scripts deleted. Check the AOSS console's Serverless generation field after apply to confirm what you actually deployed - if it shows Classic, the workaround didn't run.

Cleanup

make destroy

Verify no leftover spend with:

aws --region us-east-1 s3vectors list-vector-buckets
aws --region us-east-1 opensearchserverless list-collections
aws --region us-east-1 rds describe-db-clusters --query 'DBClusters[].DBClusterIdentifier'

All three should return empty. If you enabled Pinecone, also delete the vector-hosting-comparison index from the Pinecone console - that's prorated daily.

License

MIT.