New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Replication Groups for Elasticache #4361

Closed
zoltrain opened this Issue Dec 17, 2015 · 46 comments

Comments

Projects
None yet
@zoltrain

zoltrain commented Dec 17, 2015

Amazon have support for Mutli-AZ deployments for Elasticache nodes, including Redis v2.8.6+.

Meaning you get automatic failover, via the master/slave replication via slave promotion.

http://docs.aws.amazon.com/AmazonElastiCache/latest/APIReference/API_CreateReplicationGroup.html

Currently according to the docs it isn't possible to create one of these via terraform, would be great if this was available for the next release.

@stack72

This comment has been minimized.

Contributor

stack72 commented Dec 17, 2015

@zoltrain just a FYI, I have started this and will finish it in a day or 2 - #4363

@zoltrain

This comment has been minimized.

zoltrain commented Dec 17, 2015

Amazing! HA Redis FTW

@aldarund

This comment has been minimized.

aldarund commented Jan 12, 2016

Need this, any updates on it?

@wolfspyre

This comment has been minimized.

wolfspyre commented Jan 13, 2016

@yogin

This comment has been minimized.

yogin commented Jan 27, 2016

Need this also, any updates/ETA?

@Felivel

This comment has been minimized.

Contributor

Felivel commented Jan 28, 2016

Any news on this?

@iwashi

This comment has been minimized.

iwashi commented Feb 13, 2016

+1!

@hatt

This comment has been minimized.

hatt commented Feb 16, 2016

+1

1 similar comment
@sarkis

This comment has been minimized.

Contributor

sarkis commented Feb 18, 2016

+1

@chrisburrell

This comment has been minimized.

chrisburrell commented Feb 29, 2016

+1 - any updates on this?

@vasiliyb

This comment has been minimized.

vasiliyb commented Mar 1, 2016

+1

4 similar comments
@bogus-py

This comment has been minimized.

bogus-py commented Mar 3, 2016

+1

@anttis

This comment has been minimized.

anttis commented Mar 21, 2016

+1

@bender-the-greatest

This comment has been minimized.

bender-the-greatest commented Mar 21, 2016

+1

@kpumuk

This comment has been minimized.

kpumuk commented Mar 21, 2016

+1

@adammeghji

This comment has been minimized.

adammeghji commented Mar 29, 2016

+1

3 similar comments
@jfromaniello

This comment has been minimized.

Contributor

jfromaniello commented Apr 1, 2016

👍

@gdxhsw

This comment has been minimized.

gdxhsw commented Apr 6, 2016

+1

@mboret

This comment has been minimized.

mboret commented Apr 8, 2016

+1

@bfallik

This comment has been minimized.

Contributor

bfallik commented Apr 8, 2016

Wow, I can't tell if these +1s are trolling or not. Do the terraform developers actually use this information to prioritize work or are we all just spamming each other uselessly?

@mtougeron

This comment has been minimized.

Contributor

mtougeron commented Apr 18, 2016

@catsby Any chance this is somewhere on Hashicorp's radar for implementation? Not being able to set a replication group is preventing us from being able to use Terraform to deploy our redis clusters in AWS.

sorry to ping you directly, I just know you've worked on some of the AWS providers :)

@Dchamard

This comment has been minimized.

Dchamard commented Apr 20, 2016

+1

@n8gard

This comment has been minimized.

n8gard commented May 17, 2016

Also very much hoping to see this. We need multi-AZ Redis capabilities and it looks like we'll have to roll our own machines rather than using ElastiCache. :(

@arozwalak

This comment has been minimized.

arozwalak commented May 31, 2016

+1

@boxrick

This comment has been minimized.

boxrick commented Jun 9, 2016

Maybe this can help someone?

resource "aws_cloudformation_stack" "default_elasticache" {
name = "${var.product}-${var.redis_dns}-${var.environment}-${var.region}-replgrp"
on_failure = "DELETE"
template_body = <<STACK
{
"Resources" : {
"${var.product}${var.redis_dns}replgrp": {
"Type" : "AWS::ElastiCache::ReplicationGroup",
"Properties" : {
"AutomaticFailoverEnabled" : "true",
"AutoMinorVersionUpgrade" : "true",
"CacheNodeType" : "${var.reddis_instance_size}",
"CacheSubnetGroupName" : "${aws_elasticache_subnet_group.default_reddis_sg.name}",
"Engine" : "redis",
"NumCacheClusters" : "2",
"Port" : "6379",
"PreferredCacheClusterAZs" : ["eu-west-1a", "eu-west-1b"],
"ReplicationGroupDescription" : "${aws_elasticache_subnet_group.default_reddis_sg.name}",
"SecurityGroupIds" : ["${aws_security_group.default.id}"],
"PreferredMaintenanceWindow" : "wed:09:25-wed:22:30",
"SnapshotRetentionLimit" : "5",
"SnapshotWindow" : "03:30-05:30"
}
}
},
"Outputs" : {
"endpoint" : {
"Description": "Redis Cluster Endpoint",
"Value" : { "Fn::GetAtt" : [ "${var.product}${var.redis_dns}replgrp", "PrimaryEndPoint.Address"]}
}
}
}
STACK
}

Then I get the cache primary cluster endpoint using IAM roles on the specific box and this command. Basically I pull in the redis_cache cluster description and get the replication group and use that to find the endpoint and finally override my config.

redis_description=$(/usr/bin/curl -fs http://169.254.169.254/latest/user-data | jq -r '.host_variables.redis_description')
redis_endpoint=$(/usr/local/bin/aws --region eu-west-1 elasticache describe-replication-groups | jq -r ".ReplicationGroups[] | select(.Description == \"${redis_description}\") | .NodeGroups[].PrimaryEndpoint.Address")

@ricoli

This comment has been minimized.

ricoli commented Jun 10, 2016

this is very much needed :)

@vshimoda

This comment has been minimized.

vshimoda commented Jun 17, 2016

+1

1 similar comment
@hSATAC

This comment has been minimized.

hSATAC commented Jun 28, 2016

+1

@aq1018

This comment has been minimized.

aq1018 commented Jul 14, 2016

It's been 8 month and nothing...? Please add support!

@ajlanghorn

This comment has been minimized.

Contributor

ajlanghorn commented Jul 27, 2016

+1

1 similar comment
@hehachris

This comment has been minimized.

hehachris commented Jul 29, 2016

+1

@MrPink

This comment has been minimized.

MrPink commented Jul 29, 2016

+2

@Farhie

This comment has been minimized.

Farhie commented Aug 1, 2016

+1

1 similar comment
@guyon

This comment has been minimized.

guyon commented Aug 3, 2016

+1

@AndrewFarley

This comment has been minimized.

AndrewFarley commented Aug 3, 2016

Just saw 0.7 rc2 go out without this feature yesterday. Redundancy is quite important for a production-level architecture using Redis. I'm use non-redundancy at the moment via terraform to begin engineering on top of ElastiCache, but I hope to get this added soon. Any way we can bump the priority of this?

@CaseyLabs

This comment has been minimized.

CaseyLabs commented Aug 8, 2016

Any news on this? @mitchellh - we have a production roll out in the pipeline that I would love to use Terraform for, and Replication Groups for Elasticache would be a piece that we need.

@roblofthouse

This comment has been minimized.

roblofthouse commented Aug 9, 2016

I'm shocked this isn't in here. +1 on getting this feature out. Currently working on a production rollout of some new systems and this is a huge blocker.

@MrPink

This comment has been minimized.

MrPink commented Aug 10, 2016

Just a tip - We have been using the Terraform CloudFormation provider to build our Elasticache clusters in production and it works fine for the time being

@AndrewFarley

This comment has been minimized.

AndrewFarley commented Aug 10, 2016

@MrPink Yea I figured that is possible but didn't spend the time to do that. Mind gist-ing some example code to save some of us some time until this is implemented in Terraform?

@petestorey26

This comment has been minimized.

petestorey26 commented Aug 10, 2016

+1

1 similar comment
@brianbenns

This comment has been minimized.

brianbenns commented Aug 11, 2016

+1

@MrPink

This comment has been minimized.

MrPink commented Aug 11, 2016

Here's the Gist for creating a Replication group with Cloudformation and controlling it with Terraform https://gist.github.com/MrPink/393840d460c0cb1caa5074d3fafd5657 - Unfortunately I have to modify it after with aws-cli to enable snapshots as its currently not possible with CF.

@jgruhl

This comment has been minimized.

jgruhl commented Aug 11, 2016

+1

@FlorinAndrei

This comment has been minimized.

FlorinAndrei commented Aug 17, 2016

This is a blocker. It should be moved higher in the priority list.

@stack72

This comment has been minimized.

Contributor

stack72 commented Aug 18, 2016

Finally got back to doing the work here - #8275 should fix it up!

@stack72

This comment has been minimized.

Contributor

stack72 commented Aug 18, 2016

Closed via #8275

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment