Managing multiple accounts, one environment per account.

[include]

How can I split and organize my sceptre into multiple repositories, one per account/environment?

Lets say I have dev, stage, prod and finally a services account for shared services.

Git repositories:

dev/
stage/
prod/
services/
templates/ (??common templates repo used across all accounts/envs??)

With this in mind, how should I organize my project?
How can I reutilize a global templates repository and how can I use this schema in a secure and wise way?

KISS ideas are welcome :)

Thanks

[1oglop1]

We are using git subtree to share common templates/hooks/resolvers.
It's not super fluid but works.

[nabeelamjad]

The way we do it is pretty similar to yours @include

├── config
│   ├── config.yaml
│   ├── dev
│   │   ├── config.yaml
│   │   └── vpc.yaml
│   ├── prod
│   │   ├── config.yaml
│   │   └── vpc.yaml
│   └── vpc.yaml
├── requirements.txt
└── templates
    └── template.json

• config/vpc.yaml contains the relevant information such as template_path and sceptre_user_data (to dynamically populate using environment variables or stack outputs)
• The stacks under dev/vpc.yaml and prod/vpc.yaml are just empty files
• dev/config.yaml and prod/config.yaml contain account information for that stack group (called environment in v1)

You can utilise stack_group_path in v2 (or environment_path in v1) to dynamically determine path.
i.e.: template_bucket_name: my-bucket-{{ environment_path.0 }}-sceptre where environment_path.0 transforms into your stack/environment name that's being ran.

[include]

Hi guys, thanks for your feedback.

@1oglop1 , I had bad experience in the past using git submodules. I try to avoid them... my bad :)

@nabeelamjad , that struct is built in the same git repo or do you have different git repos for each environment?

I think each major struct should have a specific git repository, say, prod, stage, templates or even a third party / contrib.

[1oglop1]

@include That's why we are using subtrees https://codewinsarguments.co/2016/05/01/git-submodules-vs-git-subtrees/

[nabeelamjad]

@include we divide our apps/workers/etc in separate repos, but all environments (dev/staging/prod/test) lives in the same app repo, primarily because the only difference between them is the use of a different AWS account config (and the empty config file for the stack).

[include]

I am going to replicate this at home, perhaps using this https://github.com/ingydotnet/git-subrepo/blob/master/ReadMe.pod.

Meanwhile, do you run any CI/CD on top of this? (cnf-lint and friends) in a Jenkins pipeline?

[db-beachbody]

Great conversation on a similar subject is anyone using hooks to decide which environment to deploy too? For example you may have many environments i.e. dev/qa/prod. Using the Prod key and deploying to dev would install dev to the prod account. Any idea's about managing the deployment methods or solutions to manage multiple accounts would be great.

Support for assuming roles would be cool too.

[mattvonrocketstein]

We use git submodules for common cf-templates, common policies, hooks, resolvers, etc. (Hopefully the number of submodules is reduced with sceptre v2, by having better support for pip-installable hooks/resolvers.) These submodules are used inside a parent repo, which contains one or more environment folders somewhere. Some environments seem to deserve a stand-alone repo, some environments are co-hosted, and how this part is structured comes down to externalities like whether we want CI over the environment infra or whether/how we want restrict owners/commits over the repo.

Someone always hates it, but submodules / subtrees do seem pretty inevitable when it comes to reusable templates. There's no language-level package manager for foo.json.j2 or bar.yaml, so if you like reusable code and you don't solve this problem with git, then pretty much the only hack I can think of is jamming this template data into a "python package" to install with pip (yuck)

[include]

@mattvonrocketstein I see myself using those common tools/templates under a submodule too but without refer any environment flag on them. I see this as "shared libs" which are injected into each environment. They must not impose environment behaviour; be the most agnostic possible. In fact I see environments as a number (eg. I have X number of environments) not a "string"; the biggest difference is that one of them is not supposed to be deleted/broken - prod. :)
Until here this is clean but, how should I manage, or what's the best approach to manage correctly "template" versions across all this N'th environments? eg. I'm under prod and I am referring a template on my config. I expect it to be the most stable/production ready I have in my template repo; meanwhile someone under n (dev) is playing with a new template and referring to it also. If I don't put in place any versioning referral mechanism (like folks on Terraform use x) ); this may be a problem.

prod/ (repo)
prod/config/app/foo.yaml
prod/templates/app/foo.yaml [refers to **here**]

n/ (repo)
n/config/app/foo2.yaml
n/templates/app/foo.yaml [refers to **here**]

templates/ (repo)
templates/app/foo.yaml **here**

This takes me to pick-up what you've said above: stand-alone repos and to have or not CI/CD on each env. Under our organization we will have dozens of developers writing cfn (pro's and n00bs) and deploying apps into our multiple AWS accounts. In two years I see a complete mess if we don't decouple things ups and drop "shared resources" like security groups and so. To prepare our ops team to this avalanche of good and bad changes I think one major security measure is to disallow CD to prod without any ops/human review.

x)

fetching specific version of external module from dir:

https://www.terraform.io/docs/modules/usage.html

module "consul" {
  source  = "hashicorp/consul/aws"
  version = "0.0.5"

  servers = 3
}

Fetching specific version of external module from git source

https://www.terraform.io/docs/modules/sources.html

module "vpc" {
  source = "git::https://example.com/vpc.git?ref=v1.2.0"
}

[mattvonrocketstein]

Note that terraform's approach is basically equivalent, it just abstracts away the detail of the submodule management from you for better or worse. Under the hood I think it just manages the submodule itself, checks it out during terraform init and sticks it inside .terraform/

EDIT: Thinking more about this, you could build something equivalent with sceptre using a resolver. inside config/env_name/stack_name.yaml you write template: !template_from_git repo@hash/path. when the resolver runs, it does a checkout to a temporary location (reusing cached content if possible), returns the path to the temporary location, sceptre uses the file as usual. This might have some advantages but doesn't feel like KISS.. I think I'd stick to submodules ;)