This guide covers various deployment options for Bucketeer, from local development to production environments.
- Docker Compose Deployment (Recommended for Small-Medium Companies)
- Kubernetes Deployment
- Production Considerations
- Monitoring and Observability
- Troubleshooting
Docker Compose provides the easiest way to deploy Bucketeer for small to medium-sized companies without requiring Kubernetes expertise.
-
System Requirements
- Minimum: 4GB RAM, 2 CPU cores, 20GB disk space
- Recommended: 8GB RAM, 4 CPU cores, 50GB disk space
- Docker Engine 20.10+ and Docker Compose 2.0+
-
Network Requirements
- Ports 80, 443, 3306, 6379 available
- Internet access for downloading Docker images
-
Domain Setup (Optional for production)
- DNS records pointing to your server IP
- SSL certificates (or use self-signed for development)
-
Clone the repository
git clone https://github.com/bucketeer-io/bucketeer.git cd bucketeer -
Start all services
make docker-compose-up
This command will:
- Generate development certificates if they don't exist
- Create necessary directories and configuration files
- Build Bucketeer Docker images with embedded web console
- Start all services with proper dependencies
-
Verify deployment
# Check service status make docker-compose-status # View logs make docker-compose-logs
-
Access the application
- Admin Dashboard: https://web-gateway.bucketeer.test
- API Gateway: http://localhost or https://api-gateway.bucketeer.test (HTTPS)
- Health Checks:
Customize deployment by setting environment variables:
# Use specific versions
export BUCKETEER_VERSION=v1.4.0
export MYSQL_VERSION=8.1
export POSTGRES_VERSION=18.0 # optional datawarehouse alternative
export REDIS_VERSION=7.2-alpine
# Start with custom versions
make docker-compose-up# Copy and customize environment file
cp docker-compose/env.default docker-compose/.env
# Edit .env file with your preferred versions
vim docker-compose/.env
# Deploy
make docker-compose-up# Infrastructure versions
MYSQL_VERSION=8.0 # MySQL version
POSTGRES_VERSION=18.0 # POSTGRES version
REDIS_VERSION=7-alpine # Redis version
NGINX_VERSION=1.31.9-alpine # Nginx version
# Bucketeer service versions
BUCKETEER_VERSION=localenv # Version for all services
BUCKETEER_MIGRATION_VERSION=v0.4.5 # Migration service version
BUCKETEER_WEB_VERSION=localenv # Web service version
BUCKETEER_API_VERSION=localenv # API service version
BUCKETEER_BATCH_VERSION=localenv # Batch service version
BUCKETEER_SUBSCRIBER_VERSION=localenv # Subscriber service versionFor production deployments, follow these additional steps:
Use your own TLS certificate:
make docker-compose-up (and make deploy-bucketeer for the minikube/localenv
cluster) generate self-signed certificates for the local services
(tools/dev/cert/tls.key / tls.crt) and the local PostgreSQL
(tools/dev/cert/postgres-tls.key / postgres-tls.crt) on first run. These are
gitignored local development material. Do not copy a production certificate or
key into tools/dev/cert/: nothing in that directory must ever be committed, and
production certificates do not belong in the repository checkout.
For a real deployment, keep the certificate and key outside the repository:
- Kubernetes: store them in a TLS Secret and point the chart at it instead of the
development secrets that
make deploy-bucketeercreates fromtools/dev/cert:Setkubectl create secret tls bucketeer-service-cert \ --cert=your-ssl-cert.crt --key=your-ssl-key.key -n bucketeer
tls.service.secretfor each service (web,api,batch-serverandsubscriberin the umbrella chart) and the ingresstls.bucketeerJP.secretsforwebandapito that Secret. - Docker Compose: store them in a directory outside the repository that only the
Docker user can read (
chmod 600on the key) and point every service certificate mount indocker-compose/compose.ymlat them: thetls.crt/tls.keyfile mounts ofnginx(it reads them from/etc/nginx/certs),web,api,batchandsubscriber, and thetls.crtmount ofbatch-cron, which trusts it when calling the batch service.
Use a dedicated OAuth signing key and service token:
For local development, make docker-compose-up (and make deploy-bucketeer for the
minikube/localenv cluster) generate a development OAuth key pair
(tools/dev/cert/oauth-private.pem / oauth-public.pem) and a service token
(tools/dev/cert/service-token) on first run. These are gitignored and are for
local use only — the private key can mint system-admin tokens.
The service token generator is a Go program; if Go is not installed on the host
(for example a Docker-only server running prebuilt images), make runs it inside a
golang container automatically, so no extra setup is needed.
For a real deployment, provide your own managed secrets (Helm oauth.key.secret /
serviceToken.secret) and use a non-https://localhost issuer. The old committed
development key pair is in git history, so it must never be reused in a real
deployment. To generate fresh material explicitly:
# Generate a fresh OAuth key pair (writes tools/dev/cert/oauth-private.pem and oauth-public.pem)
make -C tools/dev generate-oauth
# Generate a new service token signed by that key.
# ISSUER must be your production OAuth issuer (not https://localhost).
make generate-service-token \
ISSUER=https://your-domain.com \
EMAIL=service@your-domain.com \
OAUTH_KEY_PATH=tools/dev/cert/oauth-private.pem \
SERVICE_TOKEN_PATH=tools/dev/cert/service-tokenKeep the OAuth issuer consistent:
Token verification checks the issuer, so the ISSUER used to mint the service token
must match the issuer the services are configured with.
Set the same production URL in every place that defaults to https://localhost:
docker-compose/config/oauth-config.json (issuer) and BUCKETEER_BATCH_OAUTH_ISSUER
for Docker Compose; for Helm, both web.oauth.issuer and batch-server.oauth.issuer
in the umbrella chart (oauth.issuer in each standalone web/batch chart) — the batch
service reads its own issuer and does not inherit the web one. A mismatch causes
issuer-validation failures, not a startup refusal.
Restrict Google sign-in to your Workspace domains:
Only Google accounts with a verified email address can sign in. To additionally
limit sign-in to your own Google Workspace domains, list them under
google.allowedDomains in the OAuth configuration
(docker-compose/config/oauth-config.json for Docker Compose, web.oauth.google.allowedDomains
in the umbrella chart or oauth.google.allowedDomains in the standalone web chart for Helm),
for example ["example.com"]. The list is matched against the hosted domain (hd)
Google reports for the account, so consumer Google accounts and accounts from other
domains are rejected even when their email address looks like it belongs to your domain.
This complements BUCKETEER_WEB_EMAIL_FILTER / env.emailFilter, which filters on the
email address itself.
Disable the demo sign-in:
Set the demo sign-in to false (and never isSystemAdmin: true). The path differs per
deployment: demoSignIn.enabled in docker-compose/config/oauth-config.json for Docker
Compose, and web.oauth.demoSignIn.enabled (umbrella chart) or oauth.demoSignIn.enabled
(standalone web chart) in Helm values. The demo password is published to the browser via
env.js, so a demo system admin must never be enabled in production.
Configure CORS policy:
Edit docker-compose/config/nginx/bucketeer.conf:
# Replace localhost with your actual domain
add_header 'Access-Control-Allow-Origin' 'https://your-domain.com' always;Use Docker secrets for sensitive data:
# For MySQL
# Create secret files
mkdir -p docker-compose/secrets
echo "your-mysql-password" > docker-compose/secrets/mysql_password.txt
echo "your-mysql-root-password" > docker-compose/secrets/mysql_root_password.txt
# Database URL of the migration service: user, password, host, port and database name.
# Percent-encode special characters in the user name and password.
echo "mysql://user:password@host:port/db_name" > docker-compose/secrets/migration_db_url.txt
chmod 600 docker-compose/secrets/*.txtmake docker-compose-setup creates migration_db_url.txt for the bundled MySQL when
it is missing and never overwrites it. The migration service reads the whole URL from
this file; nothing else does, so the MySQL settings of the other services in
docker-compose/compose.yml have to match it.
# For PostgresQL
mkdir -p docker-compose/secrets
echo "your-postgres-password" > docker-compose/secrets/postgres_password.txtReference existing Kubernetes Secrets from the Helm chart:
The chart renders the credentials it receives through values (the database passwords,
the OAuth configuration with the Google client secret, the email provider
configuration, the data warehouse configuration and the subscriber processor
configuration) into Kubernetes Secret objects rather than ConfigMaps or literal
container environment values. The exception is the demo sign-in password, which
env.js publishes to the browser by design when the demo sign-in is enabled (see
above). For the database passwords, the OAuth configuration and the email
configuration you can also create the Secrets yourself and point the chart at them,
so those values do not have to appear in a values file. The same applies to the
database URL of the migration job:
- Database passwords:
global.operationalDatabase.mysql.existingSecret/existingSecretKeyandglobal.operationalDatabase.postgres.existingSecret/existingSecretKey(each service chart can override them under<chart>.env.operationalDatabase.<engine>). The password is injected through asecretKeyRef. To keep the user name out of the values file as well, leaveuserempty: the user name of the engine selected byoperationalDatabase.typeis then read from the same Secret underexistingSecretUserKey(mysql-user/postgres-userby default). When no existing Secret is set, the chart creates<service>-db-credentialsfrompass, and changingpassrolls the pods. Pods do not restart on their own when the content of an existing Secret changes. To switch a running release to an existing Secret, setexistingSecretand keeppassfor that release: the new pods read the existing Secret, and<service>-db-credentialskeeps the password so that pods created before the switch can still start (a scale-out or an aborted rollout during the switch). Removepassonce the new pods have replaced the previous ones; this rolls the pods once more. - Migration database URL:
migration.existingSecret/existingSecretKey(existingSecret/existingSecretKeyin the standalonebucketeer-migrationchart), a Secret whose key (db-urlby default) holds the complete URL, for examplemysql://user:password@host:port/db_name. Percent-encode special characters in the user name and password. Unlike the passwords above,migration.dbUrlis not rendered into a Secret: it is passed to the migration job as a plain argument, so it is readable in the Job and in the Helm release. WithexistingSecretthe job reads the URL through asecretKeyRefanddbUrlis ignored. In thebucketeerchart the migration job is a pre-install/pre-upgrade hook, so the Secret must exist in the release namespace before Helm runs:kubectl create secret generic bucketeer-migration-db-url -n bucketeer \ --from-literal db-url='mysql://user:password@host:port/db_name' - OAuth configuration:
web.oauth.existingSecret, a Secret with anoauth-config.jsonkey holding the completeoauthblock as JSON. - Email configuration:
global.email.existingSecret, a Secret with anemail-config.jsonkey holding the completeglobal.emailblock as JSON. - Slack webhook URL of the demo organization creation notifier:
subscriber.processors.demoOrganizationCreationNotifier.notifier.slack.existingSecret/existingSecretKey, a Secret whose key (slack-webhook-urlby default) holds the webhook URL. It is injected into the subscriber through asecretKeyRefand takes precedence overwebhookURL, which can then stay empty. It is only used whensubscriber.env.demoSiteEnabledistrue. Pods do not restart on their own when the content of the Secret changes.
The data warehouse configuration and the rest of the subscriber processor configuration are always rendered by the chart from values; they have no existing-Secret override.
Encrypt stored secrets (FCM service accounts and Slack webhook URLs):
Push configurations store a Firebase service-account key and notification
subscriptions store a Slack incoming-webhook URL. By default (encryption-type: none)
they are stored as they are. With aead-local, the web service encrypts them with
AES-256-GCM before writing them to the database and the web and subscriber services
decrypt them when they read them back, so the values in the push, subscription
and admin_subscription tables are unusable without the key. The key is a 32-byte
value, base64-encoded, supplied the same way as the OAuth
key: a mounted file in Docker Compose and a Kubernetes Secret in Helm. The web and
subscriber services must use the same key and key id. The batch service does not need
the key, because it receives subscriptions through the web service.
# Generate a key readable only by you (make docker-compose-setup / deploy-bucketeer
# do this on first run, writing tools/dev/cert/encryption-key, which is gitignored)
(umask 077 && openssl rand -base64 32 > tools/dev/cert/encryption-key)Docker Compose: set the type and a key id in .env (the key is mounted from
tools/dev/cert/encryption-key):
BUCKETEER_ENCRYPTION_TYPE=aead-local
BUCKETEER_ENCRYPTION_KEY_ID=key-1Helm: create the Secret and point the umbrella chart at it:
kubectl create secret generic bucketeer-encryption-key --from-file=key=tools/dev/cert/encryption-keyglobal:
encryption:
type: aead-local
keyId: key-1
secret: bucketeer-encryption-keyThe key id is recorded in every encrypted value, so stored values written with a
different key are reported instead of being silently misread; keep it stable for
the lifetime of the key. Encryption applies to values written after it is enabled;
existing rows keep their current form and remain readable until they are saved again.
Switching back to none while encrypted rows exist makes those pushes and
subscriptions unusable until the key is configured again, so treat the key like the
OAuth key: back it up and never lose it.
Update resource limits for production: The current configuration includes resource limits suitable for medium deployments:
- MySQL: 2GB RAM, 1 CPU
- PostgresQL: 2GB RAM, 1 CPU
- Web Service: 1.5GB RAM, 1 CPU
- API Service: 1GB RAM, 0.75 CPU
For high-traffic environments, increase these limits in docker-compose/compose.yml.
Configure data persistence:
# Create backup script
cat > backup-bucketeer.sh << 'EOF'
#!/bin/bash
DATE=$(date +%Y%m%d_%H%M%S)
docker exec bucketeer-mysql mysqldump -u root -p bucketeer > backup_${DATE}.sql
docker run --rm -v mysql_data:/data -v $(pwd):/backup alpine tar czf /backup/mysql_data_${DATE}.tar.gz /data
EOF
chmod +x backup-bucketeer.sh
# Run daily backups via cron
echo "0 2 * * * /path/to/backup-bucketeer.sh" | crontab -For production with custom domains:
# Update /etc/hosts on client machines (development only)
echo "YOUR_SERVER_IP your-bucketeer-domain.com" >> /etc/hosts
# Or configure proper DNS records:
# A record: your-bucketeer-domain.com -> YOUR_SERVER_IP
# A record: web-gateway.your-bucketeer-domain.com -> YOUR_SERVER_IP
# A record: api-gateway.your-bucketeer-domain.com -> YOUR_SERVER_IP# Stop all services
make docker-compose-down
# Restart specific service
docker compose -f docker-compose/compose.yml restart web
# Scale a service (if supported)
docker compose -f docker-compose/compose.yml up -d --scale api=2
# View service logs
docker compose -f docker-compose/compose.yml logs -f web
# Execute command in container
docker exec -it bucketeer-web sh# Create MySQL event tables for data warehouse
make docker-compose-create-mysql-event-tables
# If you use Postgres for data warehouse
make docker-compose-create-postgres-event-tables
# Delete E2E test data
make docker-compose-delete-data
# Reset entire environment
make docker-compose-clean
make docker-compose-up# Check resource usage
docker stats
# Monitor service health
make docker-compose-status
# Check nginx access logs
tail -f docker-compose/logs/nginx/access.logFor E2E testing and SDK integration:
# Create client API key
WEB_GATEWAY_URL=web-gateway.bucketeer.test \
WEB_GATEWAY_PORT=443 \
WEB_GATEWAY_CERT_PATH=$PWD/tools/dev/cert/tls.crt \
SERVICE_TOKEN_PATH=$PWD/tools/dev/cert/service-token \
API_KEY_NAME="production-client-$(date +%s)" \
API_KEY_PATH=$PWD/tools/dev/cert/api_key_client \
API_KEY_ROLE=SDK_CLIENT \
ENVIRONMENT_ID=production \
make create-api-key
# Create server API key
WEB_GATEWAY_URL=web-gateway.bucketeer.test \
WEB_GATEWAY_PORT=443 \
WEB_GATEWAY_CERT_PATH=$PWD/tools/dev/cert/tls.crt \
SERVICE_TOKEN_PATH=$PWD/tools/dev/cert/service-token \
API_KEY_NAME="production-server-$(date +%s)" \
API_KEY_PATH=$PWD/tools/dev/cert/api_key_server \
API_KEY_ROLE=SDK_SERVER \
ENVIRONMENT_ID=production \
make create-api-keyFor larger deployments requiring high availability and scalability, use Kubernetes with Helm:
- Kubernetes cluster (1.20+)
- Helm 3.0+
- kubectl configured
# Add required repositories (if needed for dependencies)
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
# Deploy dependencies using local/custom Bucketeer Helm charts
kubectl create namespace bucketeer
# Deploy MySQL using the custom chart and development values
helm install mysql ./manifests/localenv/dependencies/mysql -n bucketeer \
--values ./manifests/localenv/dependencies/mysql/values.yaml
# Deploy Redis using the custom chart and development values
helm install redis ./manifests/localenv/dependencies/redis -n bucketeer \
--values ./manifests/localenv/dependencies/redis/values.yaml
# Deploy Bucketeer using the custom chart
helm install bucketeer ./manifests/bucketeer -n bucketeer \
--values ./manifests/bucketeer/values.prod.yamlNote: The paths point to the Bucketeer repository's custom Helm charts for MySQL and Redis. Update the values.yaml paths if you use different configurations.
For detailed Kubernetes deployment instructions, see the Minikube development guide.
IMPORTANT: Redis in Bucketeer stores critical event data using Redis Streams, not just cache data. Never configure Redis with memory eviction policies like allkeys-lru or volatile-lru as this will cause permanent data loss.
The current Docker Compose configuration includes proper persistence settings:
- AOF (Append Only File):
--appendonly yesensures all writes are logged - RDB Snapshots: Multiple save points for data durability
- fsync Policy:
--appendfsync everysecbalances performance and durability
For production deployments:
# Add these Redis settings for maximum data safety
--maxmemory-policy noeviction # CRITICAL: Never evict data
--maxmemory 4gb # Set based on your RAM capacity
--appendfsync always # Maximum durability (slower performance)For production environments, consider:
-
Database High Availability
- Use managed MySQL services (AWS RDS, Google Cloud SQL)
- Set up MySQL master-slave replication
- Configure automated backups
-
Redis High Availability
- Use managed Redis services (AWS ElastiCache, Google Memorystore)
- Set up Redis Sentinel or Cluster mode
-
Load Balancing
- Use external load balancers (AWS ALB, Google Cloud Load Balancer)
- Configure health checks and auto-scaling
-
Container Orchestration
- Migrate to Kubernetes for production scale
- Use Docker Swarm for simpler orchestration needs
Add monitoring stack to your deployment:
# Add to docker-compose.yml
prometheus:
image: prom/prometheus:latest
ports:
- "9090:9090"
volumes:
- ./config/prometheus:/etc/prometheus
networks:
- bucketeer
grafana:
image: grafana/grafana:latest
ports:
- "3000:3000"
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
- grafana_data:/var/lib/grafana
networks:
- bucketeer-
Network Security
- Use private networks for internal communication
- Implement firewall rules
- Enable TLS everywhere
-
Access Control
- Use OAuth providers for authentication
- Implement RBAC for user management
- Regular security audits
-
Data Protection
- Encrypt data at rest
- Use secrets management systems
- Regular security updates
-
Port Conflicts
# Check which process is using a port lsof -i :3306 # Solution: Stop conflicting services or change ports
-
Certificate Issues
# Regenerate development certificates make -C tools/dev generate-tls-certificate make -C tools/dev generate-postgres-tls-certificate make -C tools/dev generate-oauth # Running containers keep the certificates they were created with # (the files are bind-mounted individually), so recreate the stack make docker-compose-down && make docker-compose-up
-
Service Health Issues
# Check service logs docker compose -f docker-compose/compose.yml logs service-name # Check resource usage docker stats # Verify network connectivity docker exec bucketeer-web ping mysql
-
Memory Issues
# Check system memory free -h # Check Docker memory usage docker system df # Clean up unused resources docker system prune -f
-
Database Optimization
-- Optimize MySQL settings in docker-compose.yml command: --default-authentication-plugin=mysql_native_password --innodb-buffer-pool-size=1G --max-connections=200
-
Redis Configuration for Data Persistence
# CRITICAL: Redis stores event data via Redis Streams, not just cache # Never use memory eviction policies like allkeys-lru for production # Current configuration ensures data persistence: command: redis-server --appendonly yes --appendfsync everysec --save 900 1 --save 300 10 --save 60 10000 # For production, consider additional settings: # --maxmemory-policy noeviction # Prevent data loss # --maxmemory 2gb # Set appropriate memory limit
-
Application Tuning
- Adjust worker processes and connection pools
- Monitor and optimize database queries
- Implement caching strategies
- Documentation: Official Docs
- GitHub Issues: Report Issues
- Community: Discord
For enterprise support and professional services, contact the Bucketeer team.