Switch over to new Redis service (on hold until 1/8)

[lbeaufort]

See #4358 and #4392

openFEC/webservices/tasks/__init__.py

Lines 59 to 84 in 8a7182e

	app.conf.update(
	broker_url=redis_url(),
	broker_use_ssl={
	'ssl_cert_reqs': ssl.CERT_NONE,
	},
	redis_backend_use_ssl={
	'ssl_cert_reqs': ssl.CERT_NONE,
	},
	imports=(
	'webservices.tasks.refresh',
	'webservices.tasks.download',
	'webservices.tasks.legal_docs',
	),
	beat_schedule=schedule,
	broker_connection_timeout=30, # in seconds
	broker_connection_max_retries=0, # for unlimited retries
	task_acks_late=False
	)
	app.conf.ONCE = {
	'backend': 'celery_once.backends.Redis',
	'settings': {
	'url': redis_url() + "?ssl=true",
	'default_timeout': 60 * 60
	}
	}

Include testing of new Redis service again prior to go live

Current redis service is expected to be removed in the beginning of April 2021. We'll want to have handled this upgrade by February of next year at the latest.

Action items

Create new aws-elasticache-redis service in each space
cf create-service aws-elasticache-redis <plan> fec-elasticache-redis

• dev (1 node): cf create-service aws-elasticache-redis redis-dev fec-elasticache-redis
• stage (3 nodes): cf create-service aws-elasticache-redis redis-3node fec-elasticache-redis
• prod (5 nodes): cf create-service aws-elasticache-redis redis-5node fec-elasticache-redis
• Wait for the creation to finish
• Update the manifests to change fec-redis to fec-elasticache-redis
• Test with a manual deploy
• Test downloads
  Stage (after release is cut 12/29/20
• Unbind apps from old service

cf unbind-service api fec-redis 
cf unbind-service celery-beat fec-redis 
cf unbind-service celery-worker fec-redis 

• Rebuild API, test downloads again
• Delete service keys, if any
• Delete old service(s)

cf delete-service fec-redis  

Prod (after release is deployed 1/5/21

• Unbind apps from old service
• Delete old service(s)

Completion criteria

We've switched to aws-elasticache-redis service in:

• dev
• stage
• prod
• Understand billing implications
  $200/month for 10-node block per https://cloud.gov/pricing/#standard-services. Using 1 for dev, 3 for stage, 5 for prod = 9 total

Next steps

Create ticket for switching production when dev/stg are ready

[lbeaufort]

No reported issues, so deleted in prod with cf delete-service fec-redis