Skip to content

Latest commit

 

History

History
1582 lines (1212 loc) · 31.4 KB

yaml-netenv.rst

File metadata and controls

1582 lines (1212 loc) · 31.4 KB

NetworkEnvironments

NetworkEnvironments are files in the top-level netenv/ directory.

NetworkEnvironments are the core of any Paco project. Every .yaml file in the netenv directory contains information about networks, applications and environments. These files define how environments are provisioned and which networks and applications will be provisioned in each one.

NetworkEnvironment files are hierarchical. They are nested many levels deep. At each node in the hierarchy a different field schema is used. The top level has several key names: network:, secrets_manager:, backup_vaults:, applications: and environments:. The network: must contain a key/value pairs that matches a NetworkEnvironment schema. The applications: and environments: are containers that hold Application and Environment schemas.

network:
    availability_zones: 2
    enabled: true
    region: us-west-2
    # more network YAML here ...

applications:
    my-paco-app:
        # more application YAML here ...
    reporting-app:
        # more application YAML here ...

environments:
    dev:
        title: Development Environment
        # more environment YAML here ...
    prod:
        title: Production Environment
        # more environment YAML here ...

The network, applications, backup_vaults and secrets_manager configuration sections hold logical configuration - this configuration does not get direclty provisioned to the cloud - it doesn't reference any environments or regions. Think of it as default configuration.

Environments are where actual cloud resources are declared to be provisioned. Environments reference the default configuration from networks, applications, backups and secrets and declare which account(s) and region(s) to provision them in.

In environments, any field from the default configuration being referenced can be overridden. This could be used for running a smaller instance size in the dev environment, enabling monitoring only in a production environment, or specifying a different git branch name for a CI/CD for each environment.

Network

The network config type defines a complete logical network: VPCs, Subnets, Route Tables, Network Gateways. The applications defined later in this file will be deployed into networks that are built from this network template.

Networks have the following hierarchy:

network:
    # general config here ...
    vpc:
        # VPC config here ...
        nat_gateway:
            # NAT gateways container
        vpn_gateway:
            # VPN gateways container
        private_hosted_zone:
            # private hosted zone config here ...
        security_groups:
            # security groups here ...

SecurityGroups have two level nested names. These can be any names, but typically the first name is the name of an application and the second name is for a resource in that application. However, other name schemes are possible to support workloads sharing the same Security Groups.

  network:
    vpc:
      security_groups:
        myapp:
          lb:
            egress:
              - cidr_ip: 0.0.0.0/0
                name: ANY
                protocol: "-1"
            ingress:
              - cidr_ip: 128.128.255.255/32
                from_port: 443
                name: HTTPS
                protocol: tcp
                to_port: 443
              - cidr_ip: 128.128.255.255/32
                from_port: 80
                name: HTTP
                protocol: tcp
                to_port: 80
          web:
            egress:
              - cidr_ip: 0.0.0.0/0
                name: ANY
                protocol: "-1"
            ingress:
              - from_port: 80
                name: HTTP
                protocol: tcp
                source_security_group: paco.ref netenv.my-paco-example.network.vpc.security_groups.app.lb
                to_port: 80

NetworkEnvironment

NetworkEnvironment

:guilabel:`NetworkEnvironment`
Field name Type Purpose Constraints Default
         

Base Schemas Deployable, Named, Title

Network

:guilabel:`Network`
Field name Type Purpose Constraints Default
availability_zones Int Availability Zones   0
aws_account PacoReference Account this Network belongs to Paco Reference to Account.  
vpc Object<VPC> VPC    

Base Schemas Deployable, Named, Title

VPC

VPC

:guilabel:`VPC`
Field name Type Purpose Constraints Default
cidr String CIDR    
enable_dns_hostnames Boolean Enable DNS Hostnames   False
enable_dns_support Boolean Enable DNS Support   False
enable_internet_gateway Boolean Internet Gateway   False
nat_gateway Container<NATGateways> |star| NAT Gateways    
peering Container<VPCPeerings> |star| VPC Peering    
private_hosted_zone Object<PrivateHostedZone> Private hosted zone    
security_groups Container<SecurityGroupSets> |star| Security Group Sets Security Groups Sets are containers for SecurityGroups containers.  
segments Container<Segments> |star| Segments    
vpn_gateway Container<VPNGateways> |star| VPN Gateways    

Base Schemas Deployable, Named, Title

VPCPeerings

Container for VPCPeering objects.

:guilabel:`VPCPeerings`|bars| Container<VPCPeering>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

VPCPeering

VPC Peering

:guilabel:`VPCPeering`
Field name Type Purpose Constraints Default
network_environment PacoReference Network Environment Reference Paco Reference to NetworkEnvironment.  
peer_account_id String Remote peer AWS account Id    
peer_region String Remote peer AWS region    
peer_role_name String Remote peer role name    
peer_vpcid String Remote peer VPC Id    
routing List<VPCPeeringRoute> |star| Peering routes    

Base Schemas Deployable, Named, Title

VPCPeeringRoute

VPC Peering Route

:guilabel:`VPCPeeringRoute`
Field name Type Purpose Constraints Default
cidr String CIDR IP A valid CIDR v4 block or an empty string  
segment PacoReference Segment Paco Reference to Segment.  

NATGateways

Container for NATGateway objects.

:guilabel:`NATGateways`|bars| Container<NATGateway>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

NATGateway

NAT Gateway

:guilabel:`NATGateway`
Field name Type Purpose Constraints Default
availability_zone String Availability Zones to launch instances in. Can be 'all' or number of AZ: 1, 2, 3, 4 ... all
default_route_segments List<PacoReference> Default Route Segments Paco Reference to Segment.  
ec2_instance_type String EC2 Instance Type   t2.nano
ec2_key_pair PacoReference EC2 key pair Paco Reference to EC2KeyPair.  
security_groups List<PacoReference> Security Groups Paco Reference to SecurityGroup.  
segment PacoReference Segment Paco Reference to Segment.  
type String NAT Gateway type   Managed

Base Schemas Deployable, Named, Title

VPNGateways

Container for VPNGateway objects.

:guilabel:`VPNGateways`|bars| Container<VPNGateway>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

VPNGateway

VPN Gateway

:guilabel:`VPNGateway`
Field name Type Purpose Constraints Default
         

Base Schemas Deployable, Named, Title

PrivateHostedZone

Private Hosted Zone

:guilabel:`PrivateHostedZone`
Field name Type Purpose Constraints Default
name String Hosted zone name    
vpc_associations List<String> List of VPC Ids    

Base Schemas Deployable

Segments

Container for Segment objects.

:guilabel:`Segments`|bars| Container<Segment>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

Segment

Segment

:guilabel:`Segment`
Field name Type Purpose Constraints Default
az1_cidr String Availability Zone 1 CIDR    
az2_cidr String Availability Zone 2 CIDR    
az3_cidr String Availability Zone 3 CIDR    
az4_cidr String Availability Zone 4 CIDR    
az5_cidr String Availability Zone 5 CIDR    
az6_cidr String Availability Zone 6 CIDR    
internet_access Boolean Internet Access   False

Base Schemas Deployable, Named, Title

SecurityGroupSets

Container for SecurityGroups objects.

:guilabel:`SecurityGroupSets`|bars| Container<SecurityGroups>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

SecurityGroups

Container for SecurityGroup objects.

:guilabel:`SecurityGroups`|bars| Container<SecurityGroup>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

SecurityGroup

AWS Resource: Security Group

:guilabel:`SecurityGroup`
Field name Type Purpose Constraints Default
egress List<EgressRule> Egress Every list item must be an EgressRule  
group_description String Group description Up to 255 characters in length  
group_name String Group name Up to 255 characters in length. Cannot start with sg-.  
ingress List<IngressRule> Ingress Every list item must be an IngressRule  

Base Schemas Deployable, Named, Title

EgressRule

Security group egress

:guilabel:`EgressRule`
Field name Type Purpose Constraints Default
destination_security_group PacoReference|String Destination Security Group Reference A Paco reference to a SecurityGroup Paco Reference to SecurityGroup. String Ok.  

Base Schemas SecurityGroupRule, Name

IngressRule

Security group ingress

:guilabel:`IngressRule`
Field name Type Purpose Constraints Default
source_security_group PacoReference|String Source Security Group Reference A Paco Reference to a SecurityGroup Paco Reference to SecurityGroup. String Ok.  

Base Schemas SecurityGroupRule, Name

Applications

Applications define a collection of AWS resources that work together to support a workload.

Applications specify the sets of AWS resources needed for an application workload. Applications contain a mandatory groups: field which is container of ResrouceGroup objects. Every AWS resource for an application must be contained in a ResourceGroup with a unique name, and every ResourceGroup has a Resources container where each Resource is given a unique name.

Attention!

ResourceGroups and individual Resources both have an order field. When resources are created, they will be created based on the value of these order fields. First, the ResrouceGroup order is followed. The lowest order for a ResourceGroup will indicate that all those resources need to be created first, and then each Resource within a group will be created based on the order it is given.

In the example below, the groups: contain keys named cicd, website and bastion. In turn, each ResourceGroup contains resources: with names such as cpbd, cert and alb.

applications:
    my-paco-app:
        enabled: true
        groups:
            cicd:
                type: Deployment
                resources:
                    cpbd:
                        # CodePipeline CI/CD
                        type: DeploymentPipeline
                        # configuration goes here ...
            website:
                type: Application
                resources:
                    cert:
                        type: ACM
                        # configuration goes here ...
                    alb:
                        # Application Load Balancer (ALB)
                        type: LBApplication
                        # configuration goes here ...
                    webapp:
                        # AutoScalingGroup (ASG) of web server instances
                        type: ASG
                        # configuration goes here ...
            bastion:
                type: Bastion
                resources:
                    instance:
                        # AutoScalingGroup (ASG) with only 1 instance (self-healing ASG)
                        type: ASG
                        # configuration goes here ...

ApplicationEngines

A container for Application Engines

:guilabel:`ApplicationEngines`|bars| Container<ApplicationEngine>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

Application

An Application is groups of cloud resources to support a workload.

:guilabel:`Application`
Field name Type Purpose Constraints Default
         

Base Schemas DNSEnablable, Deployable, ApplicationEngine, Monitorable, Named, Notifiable, Title

ResourceGroups

A container of Application ResourceGroup objects.

:guilabel:`ResourceGroups`|bars| Container<ResourceGroup>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

ResourceGroup

A group of Resources to support an Application.

:guilabel:`ResourceGroup`
Field name Type Purpose Constraints Default
dns_enabled Boolean      
order Int |star| The order in which the group will be deployed    
resources Container<Resources> |star|      
title String Title    
type String |star| Type    

Base Schemas Deployable, Named

Resources

A container of Resources to support an Application.

:guilabel:`Resources`
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

Environments

Environments define where actual cloud resources are to be provisioned. As Environments copy all of the defaults from network, applications, backups and secrets_manager config in the same NetworkEnvironment file.

The top level environments: container is simply a name and a title. This defines logical names for each environment.

environments:

    dev:
        title: Development

    staging:
        title: Staging and QA

    prod:
        title: Production

Environments contain EnvironmentRegions. The name of an EnvironmentRegion must match a valid AWS region name. The special default name is also available, which can be used to override config for a whole environment, regardless of region.

The following example enables the applications named marketing-app and sales-app into all dev environments by default. In us-west-2 this is overridden and only the sales-app would be deployed there.

environments:

    dev:
        title: Development
        default:
            applications:
                marketing-app:
                    enabled: true
                sales-app:
                    enabled: true
        us-west-2:
            applications:
                marketing-app:
                    enabled: false
        ca-central-1:
            enabled: true

Environment

Environment

:guilabel:`Environment`
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

EnvironmentDefault

Default values for an Environment's configuration

:guilabel:`EnvironmentDefault`
Field name Type Purpose Constraints Default
applications Container<ApplicationEngines> |star| Application container    
network Container<Network> Network    
secrets_manager Container<SecretsManager> Secrets Manager    

Base Schemas RegionContainer, Named, Title

EnvironmentRegion

An actual provisioned Environment in a specific region. May contains overrides of the IEnvironmentDefault where needed.

:guilabel:`EnvironmentRegion`
Field name Type Purpose Constraints Default
         

Base Schemas RegionContainer, EnvironmentDefault, Deployable, Named, Title

Secrets

SecretsManager

Secrets Manager contains SecretManagerApplication objects.

:guilabel:`SecretsManager`|bars| Container<SecretsManagerApplication>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

SecretsManagerApplication

Container for SecretsManagerGroup objects.

:guilabel:`SecretsManagerApplication`|bars| Container<SecretsManagerGroup>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

SecretsManagerGroup

Container for SecretsManagerSecret objects.

:guilabel:`SecretsManagerGroup`|bars| Container<SecretsManagerSecret>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

SecretsManagerSecret

Secret for the Secrets Manager.

:guilabel:`SecretsManagerSecret`
Field name Type Purpose Constraints Default
account PacoReference Account to provision the Secret in Paco Reference to Account.  
generate_secret_string Object<GenerateSecretString> Generate SecretString object    

Base Schemas Deployable, Named, Title

GenerateSecretString

:guilabel:`GenerateSecretString`
Field name Type Purpose Constraints Default
exclude_characters String A string that includes characters that should not be included in the generated password.    
exclude_lowercase Boolean The generated password should not include lowercase letters.   False
exclude_numbers Boolean The generated password should exclude digits.   False
exclude_punctuation Boolean The generated password should not include punctuation characters.   False
exclude_uppercase Boolean The generated password should not include uppercase letters.   False
generate_string_key String The JSON key name that's used to add the generated password to the JSON structure.    
include_space Boolean The generated password can include the space character.    
password_length Int The desired length of the generated password.   32
require_each_included_type Boolean The generated password must include at least one of every allowed character type.   True
secret_string_template String A properly structured JSON string that the generated password can be added to.    

Base Schemas Deployable

Backups

AWS Backup can be provisioned with the backup_vaults:. This is a container of BackupVaults. Each BackupVault can contain BackupPlans which are further composed of a BackupRules and BackupSelections.

backup_vaults:
  accounting:
    enabled: false
    plans:
      ebs_daily:
        title: EBS Daily Backups
        enabled: true
        plan_rules:
          - title: Backup EBS volumes once a day
            schedule_expression: cron(0 8 ? * * *)
            lifecycle_delete_after_days: 14
        selections:
          - title: EBS volumes tagged with "backup-accounting: daily"
            tags:
              - condition_type: STRINGEQUALS
                condition_key: backup-accounting
                condition_value: daily
      database_weekly:
        title: Weekly MySQL Backups
        enabled: true
        plan_rules:
          - title: Rule for Weekly MySQL Backups
            schedule_expression: cron(0 10 ? * 1 *)
            lifecycle_delete_after_days: 150
        selections:
          - title: Database resource selection
            resources:
              - paco.ref netenv.mynet.applications.accounting.groups.app.resources.database

BackupVaults must be explicity referenced in an environment for them to be provisioned.

environmnets:
  prod:
    ca-central-1:
      backup_vaults:
        accounting:
          enabled: true

BackupVaults

Container for BackupVault objects.

:guilabel:`BackupVaults`|bars| Container<BackupVault>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

BackupVault

An AWS Backup Vault.

:guilabel:`BackupVault`
Field name Type Purpose Constraints Default
notification_events List<String> Notification Events Each notification event must be one of BACKUP_JOB_STARTED, BACKUP_JOB_COMPLETED, RESTORE_JOB_STARTED, RESTORE_JOB_COMPLETED, RECOVERY_POINT_MODIFIED  
notification_group String Notification Group    
plans Container<BackupPlans> Backup Plans    

Base Schemas Resource, DNSEnablable, Deployable, Named, Title, Type

BackupPlans

Container for BackupPlan objects.

:guilabel:`BackupPlans`|bars| Container<BackupPlan>
Field name Type Purpose Constraints Default
         

Base Schemas Named, Title

BackupPlan

AWS Backup Plan

:guilabel:`BackupPlan`
Field name Type Purpose Constraints Default
plan_rules List<BackupPlanRule> |star| Backup Plan Rules    
selections List<BackupPlanSelection> Backup Plan Selections    

Base Schemas Resource, DNSEnablable, Deployable, Named, Title, Type

BackupPlanRule

:guilabel:`BackupPlanRule`
Field name Type Purpose Constraints Default
copy_actions List<BackupPlanCopyActionResourceType_> Copy actions   []
lifecycle_delete_after_days Int Delete after days    
lifecycle_move_to_cold_storage_after_days Int Move to cold storage after days If Delete after days value is set, this value must be smaller  
schedule_expression String Schedule Expression Must be a valid Schedule Expression.  

Base Schemas Named, Title

BackupPlanSelection

:guilabel:`BackupPlanSelection`
Field name Type Purpose Constraints Default
resources List<PacoReference> Backup Plan Resources Paco Reference to Interface.  
tags List<BackupSelectionConditionResourceType> List of condition resource types    
title String |star| Title    

BackupSelectionConditionResourceType

:guilabel:`BackupSelectionConditionResourceType`
Field name Type Purpose Constraints Default
condition_key String |star| Tag Key    
condition_type String |star| Condition Type String Condition operator must be one of: StringEquals, StringNotEquals, StringEqualsIgnoreCase, StringNotEqualsIgnoreCase, StringLike, StringNotLike.  
condition_value String |star| Tag Value