Skip to content
Site Reliability Engineering / DevOps SaltStack configuration files
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.git-crypt
.github
bin
bootstrap-aws
docs
pillars
states
.cc-metadata.yml
.gitattributes
.gitignore
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md

README.md

sre-salt-prime

Site Reliability Engineering / DevOps SaltStack configuration files

Code of Conduct

CODE_OF_CONDUCT.md:

The Creative Commons team is committed to fostering a welcoming community. This project and all other Creative Commons open source projects are governed by our Code of Conduct. Please report unacceptable behavior to conduct@creativecommons.org per our reporting guidelines.

Contributing

See CONTRIBUTING.md.

Development Notes

  • Avoid insecure repository clones: This repository includes encrypted secrets. Do not run git-crypt unlock on clones that are not otherwise secured (ex. strong login password, disk encryption).
  • Avoid editing the base environment: The base environment is configured to prevent commit and push actions. Please use your development environment and pull the changes to base.
  • Sign your commits:
    • The master branch has the Require signed commits (Include administrators) GitHub branch protection enabled.
    • Ensure you are using RemoteForward in your SSH configuration to forward your GnuPG agent to salt-prime (see the example configuration, under Setup, below).
    • Ensure you have configured your newly cloned repository to sign commits (see the git config command, under Setup, below).

Setup

  • SSH connection information: example local/laptop ~/.ssh/config configugration:
    Host bastion-us-east-2
        HostName bastion-us-east-2.creativecommons.org
        User ARTHUR
    
    Host salt-prime
        HostName 10.22.11.11
        ProxyJump bastion-us-east-2
        RemoteForward /run/user/4242/gnupg/S.gpg-agent /Users/ARTHUR/.gnupg/S.gpg-agent.extra
        User ARTHUR
    
    Host *
        ServerAliveCountMax 60
        ServerAliveInterval 30
        TCPKeepAlive no
    
    • Assumes remote username ARTHUR and remote uid 4242. Replace these values in your own local/laptop configuration.
    • ProxyJump allows you to ssh salt-prime from your local/laptop.
    • RemoteForward allows you to sign your commits.
  • Setup your development repository on salt-prime:
    1. Clone repository to /srv with your username. For example:
      cd /srv
      git clone git@github.com:creativecommons/sre-salt-prime.git ${USER}
    2. Setup your newly cloned repository.
      1. Configure commit signing:
        cd /srv/${USER}
        git config user.email YOUR_EMAIL
        git config user.signingkey YOUR_GPG_ID
        git config commit.gpgsign true
      2. Unlock encrypted secrets:
        cd /srv/${USER}
        git-crypt unlock
    3. Specify the environment when you test changes. For example:
      sudo salt \* state.highstate saltenv=${USER} test=True
      • use --state-verbose=True to see successes
      • use --state-output=full_id to see full detail of successes
      • use --log-level=debug --log-file-level=warning to see debug messages (without logging those debug messages, which may contain secrets, to the log file)

Goals

  • Use AWS well, but avoid technologies that are create AWS lock-in (ex. Confidant)
  • Salt Prime must not contain any exclusive data (use Git)
  • Git repository must not contain any unencrypted secrets
  • Git repository commits must be signed
  • A compromised minion must not be able to escalate access
    • SysAdmins must not forward their SSH agent
    • Must not reuse application passwords (ex. Prod and Dev databases must have different passwords)
    • Pillar data must be restricted by Minion ID based classification
      • The only grain which can be safely used is grains['id'] which contains the Minion ID. (FAQ Q.21)

Decisions

  • Amazon Web Services (AWS)
    • Creative Commons is already using it and staff are familiar with it
    • Features allow security (ex. screened subnets, security groups policies)
    • Features allows Infrastructure as Code
    • us-east-2
      • cost effective
      • avoid conflict/collision over region limited resources (ex. ElasticIPs)
  • Debian 9 (Stretch)
    • Free/Open Source
    • Debian Stable
    • Creative Commons is already using it and staff are familiar with it
  • git-crypt - transparent file encryption in git
    • Free/Open Source
    • Performance: files are decrypted in the checked out repository
    • Security: automatic encryption and directory based filters minimize the chance of unencrypted secrets being pushed to GitHub
  • SaltStack
    • Free/Open Source
    • Performance
    • Creative Commons is already using it and staff are familiar with it
    • Version: 2019.2.0

Host Classification

Minions are added and configured from salt-prime with the following Minion ID schema: HST__POD__LOC (host/role__pod/group__location). Examples:

  • bastion__core__us-east-2
  • salt-prime__core__us-east-2
  • chapters__prod__us-east-2
  • chapters__stage__us-east-2

Like Apache2, SaltStack pillar data uses a last declared wins model. This repository uses (from least specific to most specific):

  1. 1_LOC (location)
  2. 2_POD (pod/group)
  3. 3_HST (host/role)
  4. 4_POD__LOC (pod/group and location)
  5. 5_HST__POD (host/role and pod/group)

This method of setting least-specific to most-specific pillar data was inspired by Hiera.

See docs/Orchestration.md for how these classification parts are used with orchestration.

Orchestration

See docs/Orchestration.md.

References

SaltStack

Best Practices

Module Documentation

Orchestration Documentation

See docs/Orchestration.md.

WordPress

See docs/WordPress.md.

Bootstrap

See bootstrap-aws/README.md.

Formula Repositories

License

You can’t perform that action at this time.