Skip to content
AWS Quick Start Team
Branch: master
Clone or download
sasgwb and vsnyc AWS Quickstart for SAS Viya 1.4 (#12)
* (MERCURY-117) add utility script for security group mgmt
- add deployment-scripts/addaccess.sh to ansible-controller
- add ec2 security group permissions to ansible controller

Change-Id: I5d844aca79d5c27df8e482caeca592fb406b221a

* (MERCURY-161) add scripts to start/stop VMs
- add start|stop_viya_vms.sh to ansible controller
- add permissions to the ansible controller instance profile
- use more recent virk 3.4 commit (one that includes the service-management playbooks)
- set dummy branch on virk checkout (to avoid "headless" branch messages)

Change-Id: If0a279f24f6b1d43414fa3408e069baab1484976

* (MERCURY-166) remove viya logs

Change-Id: I8f373d2a02b7e9ae628d5ed1f474c69817df90a5

* (MERCURY-161) fix hardcoded region name

Change-Id: Ieab8cf8269ac20f86bf67bee006434851c0a76df

* (MERCURY-157) add "small" topology option
- add DeploymentSize parameter
- add ViyaServices instance
- add IsSmall/IsMedium conditions - create VMs conditionally
- replace hostnames with hostgroups in ansible playpens
- add additional hostgroups to ansible inventory
- move adding elb listeners to install.sh
- parametrize ansible inventory creation
- parametrize ansible vars.yml creation
- remove update.vars.file.yml (cruft)
- remove sas workers code remnants
- shorten known_hosts file initialization code
- do other small code formatting fixes

Change-Id: I29d5a8792f34ec8264f4eb6140b04f42f0b6edb0

* (MERCURY-182) An error occurred due to instances not being created

This commit fixes an issue caused by the loop invariant used when
waiting for the Viya VM nodes to be created.

In install.sh, the "Checking Viya VMs" loop has an invariant which
is always false, thus the body of the loop never executes and the
loop can exit even when all Viya VMs are not yet created. As the code
following that loop assumes the Viya VMs are created, this causes
an error.

This commit modifies the invariant such that it will not exit the loop
until ALL the Viya VMs are created.

Change-Id: I563a7186a44c5d81b237c2f42466d065e45d6db9

* (AWSVIYA-5) add data agent scripts

Change-Id: I3f168d2a54f58c159267cf9a544776ff08ac816d

* (AWSVIYA-64) support 32 core cas instance

Change-Id: Ie4c05129c69dc96d0c7a9b1cf49f58e6de1b7033

* (AWSVIYA-3) switch to "small" topology.
change default services size to 4xlarge
This change was already in place. Just needed to change the hardcoded defaults.
Full removal of multi-VM architecture will follow in AWSVIYA-63 when we add the sizing option.

Change-Id: Iff55b6a1a1f64f731c3ce9f1b5b1629189324261

* (MERCURY-192) common refactor
add nfs mounts and key exchange
start using "commons" submodule

Change-Id: Ic9d5db449ac358865743925498ffe7bd591a3c6d

* (AWSVIYA-63) add ServicesInstanceSize parameter
- remove DeploymentSize code artifacts

Change-Id: I08170195ae83cc935bd5e3277cd6aa9b7f9be28b

* (MERCURY-192) add readiness_flags
create /etc/hosts header

Change-Id: Icefb782c5094fdd3c803475dbb404a41210becb8

* (MERCURY-192) use file tree to download all files
- also: start using group_vars

Change-Id: Ibf515d104c31771cab294cfe42b02ac9fa00c5a4

* (MERCURY-192) set up hosts routing and log directory
- move ansible.cfg into common
- move "set hosts" into common
- move seed_known_hosts_file into common
- move VM readiness check into common
- disable install.sh

Change-Id: Ic20eaef6c740523911e288d2d3d4f235bdb80654

* (MERCURY-192) add disk setup
- invoke predeployment.yml

Change-Id: Ie6fa39b24f627c5602ab037c9afbd347a2aa124c

* (MERCURY-192) improve disk setup
- move scripts/disks_ephemeral into common: mount_cascache role
- set deployment log location to /var/log/sas/install
- remove predeployment steps from isntall.sh

Change-Id: I58f341544f716f53083bda3d2bcbb12c55b3c9be

* (MERCURY-192) - implement suggestions from previous push
- replace INSTALL_USER with ansible_user_id (require all scripts to be run by the install user)
- do not merge common/ files into parent directory

Change-Id: I9d846998f0aa0d98f430e005ef2d319e9c974352

* (MERCURY-192) move openldap into common
- move base64 encoding into cf template
- move conditional (only if sasuser pw is set) into cf template

Change-Id: Id733099b932aec030c0235ecdc8d5570c6ca82d9

* (MERCURY-192) add prepare_deployment.yml playbook invocation
- rename predeploy (playbook and role subdir) to prepare_nodes

Change-Id: Ic824b4d1a2e2049ba0550dfc5c710eec07557ceb

* (MERCURY-192) move virk to common
- also: fix download_file_tree.sh to set permissions on parent dirs

Change-Id: Ic190cb8979055ff4b2d7b0d3a0a14ea35e2b8efa

* (MERCURY-192) add mirror

Change-Id: I52e501420465899dd78fd243a513ed0369909c13

* (MERCURY-192) download soe file and create playbook

Change-Id: I82e460321e1696ed74a277391909920d4b5adaac

* (MERCURY-192) add playbook prep and viya install invocation

Change-Id: Icac0b01932151d5e87899cd645869dcf9a353df7

* (MERCURY-192) move ssl termination
from elb to httpproxy if not cert given.
IAW do not create and delete self signed cert.
The ELB can use TCP/443 instead of HTTPS/443.

Change-Id: I5eed70246d12120d263d925b356d67a88f76e1fa

* (MERCURY-192) add backup vault dir

Change-Id: I2f8df3f4cb137092a6c97d41cea15bea17c53f75

* (MERCURY-192) - virk invocation
- move virk invocation up to the top level (IAAS template)
- need to also move backup vault setup (because it needs the sas user)
  from prepare_nodes to new post_install

Change-Id: I05a0cb7712cfb62eb421187010e929556f02b955

* (MERCURY-192) - implement roles_path lookthrough
- move 'roles' directory out of 'playbooks'
- add 'ansible' directory
- copy host-specific roles into 'ansible/roles'
- also: add log files to cloudwatch config
- move download_file_tree.sh to common

Change-Id: I7707c038fe5a6aade14b040a95f7eaabfc9df4f7

* (MERCURY-192) add early addtl paramater verification
- make ansible controller policies conditional where possible
- add sns messaging

Change-Id: I827309302e27f45dc9a0331f25555cc95478ff64

* (AWSVIYA-103) add sns messages and parm validation
- replace common/download/download_file_tree.sh with faster code
- add pystache file postprocessing
- add send_sns_message.sh script
- remove ssl cert and dns verification (better and faster by CFN)
- add Mirror validation in validate_parameters.sh
- make some ansible controller iam policies conditional
- clean up ansible controller policies syntax

Change-Id: I62fe1516c735b17facadf477039a902c788532f8

* (AWSVIYA-103) add volume tagging
- move mount_disk task from common and add tagging
- template: add permssions for tagging to viya instances role
- clean up permissions syntax

Change-Id: Ia409770dd5afba535380e73fbdb6147f57ae146a

* (AWSVIYA-103) stop/start vm scripts
- move virk under the viya install (required by the virk stop/start playbooks)
- make stop/start scripts parameterless (get all parms from instance metadata)

Change-Id: Ie26833ab9cc0476d0f8bc451400cbe19b2a05af9

* (AWSVIYA-103) update dataagent scriptUse the the more generic version of the dataprep2dataagent playbook
the change in the inventory hosts definition made this necessary.

Change-Id: Id573af863fe582794d341ebebaf6e7488200d486

* (AWSVIYA-43) protect/remove passwords
- some scripts need the admin pw. Set all scripts to 700
- the viya ansible debug log has a dump of sitedefault.yml - remove -v from main playbook invocation;

Change-Id: I4f3e9ce06c8d20c42c46adbebaf24b297fc55a54

* (AWSVIYA-103) adjust recover_cascontroller.sh
- use new install playbooks in script
- replace values in script with pystache
- reenable PlacementGroup
- also: move ViyaVersion from prepare_nodes to prepare_deployment
- also: fix failure message invocation

Change-Id: Ic24dc99c78a508e105c0064370a348d5ccd59217

* (AWSVIYA-103) refactor addaccess.sh script
- also: clean up and comment ansible controller Role policies

Change-Id: Idce452b8186f3c84f83386dbe8e9f0991ea3fb7e

* (AWSVIYA-59) start_viya_vms.sh: stop script on error

Change-Id: I1c7590e75f67c6cb544bae1fc57b0764ad954ff2

* (AWSVIYA-103) add self-signed cert for elb
- add post_deployment/configure_cert role
- add post_deployment plabook
- re-add DelCert lambda function
- add ansible controller policy for iam and qelb actions
- call post_deployment playbook from template

Change-Id: I5590f9d1deeb3e1c82d6e1286841e045739291b2

* (MERCURY-103) clean up file permissions on ansible controller:
- make sure everything is read|write-only for the ec2-user
- remove unused script files
- also: fix cloud-init RC handling for ansible controller

Change-Id: I64484b59759848ab73a5390266c8f67767aad58f

* (AWSVIYA-103) cas_cache permissions for r4
- re-apply ownership/permissions are file mount (this only affects cas cache for r4
the other volumes get their permissions correctly from VIRK)
- make sure to use the ec2-user instead of root (ansible_user_id uses the value of become_user)
- also: add describeTags permission so the volume tagging works (regression instroduced by the permission cleanup in earlier push)

Change-Id: Ie99913b9fc4322affa3e7c55be35d70e74d232d9

* (AWSVIYA-103) fix storage size in topology diagram
- also: lowercase the "i" in "i3" for the cas controller

Change-Id: I81ef645d502f587add13b8262993b9a0625584dd

* (AWSVIYA-103) re-add sns script
turns out we need to keep the sns script because it is called unconditionally,
and the check for operatoremail resp. snstopic is within the script.
That is  actually ok, else we would have checks each time the script is called

Change-Id: I6541b9e370956a7e31db976e2a024d2996561a1f

* (AWSVIYA-112) fix vm restart script
the check for full vm initialization actually fails the script
if the sshd is not yet up (which it is in most cases).
Needed to disable "set -e" for the time of the check.

Change-Id: I40e6381edc81da6005daaf9d564fb082ec14acb1

* (AWSVIYA-63) fix missed host in addldapuser.sh script

Change-Id: I33742cf1282600a6230c2884307e1cf9992be9a8

* point common submodule to github location

* (AWSVIYA-118) move EIP assignment into template

* (AWSVIYA-103) include common code from AWSVIYA-1.4 tag

* (AWSVIYA-125) dataprep utility authentication and error check
use https and -k for sas-admin commands
catch errors in the ansible task;

* (AWSVIYA-103) remove .git directory from common code

* (AWSVIYA-127) mask sasadmin password when passing into prepare_deployment playbook
1 - call prepare_deployment with base64 encoded value for sasadmin pw
2 - generate the recover_cascontroller.sh script with the base64 encoded value
    That script, if run, calls the openldap playbook. It used to get the pw values
    in open code. Now we pass them in already encoded, so there is no need to
    do the encoding in bash

* Removed hard-coded sa-east-1 test

* (MERCURY-117) add utility script for security group mgmt
- add deployment-scripts/addaccess.sh to ansible-controller
- add ec2 security group permissions to ansible controller

Change-Id: I5d844aca79d5c27df8e482caeca592fb406b221a

* (MERCURY-161) add scripts to start/stop VMs
- add start|stop_viya_vms.sh to ansible controller
- add permissions to the ansible controller instance profile
- use more recent virk 3.4 commit (one that includes the service-management playbooks)
- set dummy branch on virk checkout (to avoid "headless" branch messages)

Change-Id: If0a279f24f6b1d43414fa3408e069baab1484976

* (MERCURY-157) add "small" topology option
- add DeploymentSize parameter
- add ViyaServices instance
- add IsSmall/IsMedium conditions - create VMs conditionally
- replace hostnames with hostgroups in ansible playpens
- add additional hostgroups to ansible inventory
- move adding elb listeners to install.sh
- parametrize ansible inventory creation
- parametrize ansible vars.yml creation
- remove update.vars.file.yml (cruft)
- remove sas workers code remnants
- shorten known_hosts file initialization code
- do other small code formatting fixes

Change-Id: I29d5a8792f34ec8264f4eb6140b04f42f0b6edb0

* (MERCURY-166) remove viya logs

Change-Id: I8f373d2a02b7e9ae628d5ed1f474c69817df90a5

* (MERCURY-161) fix hardcoded region name

Change-Id: Ieab8cf8269ac20f86bf67bee006434851c0a76df

* (MERCURY-182) An error occurred due to instances not being created

This commit fixes an issue caused by the loop invariant used when
waiting for the Viya VM nodes to be created.

In install.sh, the "Checking Viya VMs" loop has an invariant which
is always false, thus the body of the loop never executes and the
loop can exit even when all Viya VMs are not yet created. As the code
following that loop assumes the Viya VMs are created, this causes
an error.

This commit modifies the invariant such that it will not exit the loop
until ALL the Viya VMs are created.

Change-Id: I563a7186a44c5d81b237c2f42466d065e45d6db9

* (AWSVIYA-5) add data agent scripts

Change-Id: I3f168d2a54f58c159267cf9a544776ff08ac816d

* (MERCURY-192) common refactor
add nfs mounts and key exchange
start using "commons" submodule

Change-Id: Ic9d5db449ac358865743925498ffe7bd591a3c6d

* (AWSVIYA-64) support 32 core cas instance

Change-Id: Ie4c05129c69dc96d0c7a9b1cf49f58e6de1b7033

* (AWSVIYA-63) add ServicesInstanceSize parameter
- remove DeploymentSize code artifacts

Change-Id: I08170195ae83cc935bd5e3277cd6aa9b7f9be28b

* (AWSVIYA-3) switch to "small" topology.
change default services size to 4xlarge
This change was already in place. Just needed to change the hardcoded defaults.
Full removal of multi-VM architecture will follow in AWSVIYA-63 when we add the sizing option.

Change-Id: Iff55b6a1a1f64f731c3ce9f1b5b1629189324261

* (MERCURY-192) add readiness_flags
create /etc/hosts header

Change-Id: Icefb782c5094fdd3c803475dbb404a41210becb8

* (MERCURY-192) use file tree to download all files
- also: start using group_vars

Change-Id: Ibf515d104c31771cab294cfe42b02ac9fa00c5a4

* (MERCURY-192) set up hosts routing and log directory
- move ansible.cfg into common
- move "set hosts" into common
- move seed_known_hosts_file into common
- move VM readiness check into common
- disable install.sh

Change-Id: Ic20eaef6c740523911e288d2d3d4f235bdb80654

* (MERCURY-192) add disk setup
- invoke predeployment.yml

Change-Id: Ie6fa39b24f627c5602ab037c9afbd347a2aa124c

* (MERCURY-192) improve disk setup
- move scripts/disks_ephemeral into common: mount_cascache role
- set deployment log location to /var/log/sas/install
- remove predeployment steps from isntall.sh

Change-Id: I58f341544f716f53083bda3d2bcbb12c55b3c9be

* (MERCURY-192) - implement suggestions from previous push
- replace INSTALL_USER with ansible_user_id (require all scripts to be run by the install user)
- do not merge common/ files into parent directory

Change-Id: I9d846998f0aa0d98f430e005ef2d319e9c974352

* (MERCURY-192) move openldap into common
- move base64 encoding into cf template
- move conditional (only if sasuser pw is set) into cf template

Change-Id: Id733099b932aec030c0235ecdc8d5570c6ca82d9

* (MERCURY-192) add prepare_deployment.yml playbook invocation
- rename predeploy (playbook and role subdir) to prepare_nodes

Change-Id: Ic824b4d1a2e2049ba0550dfc5c710eec07557ceb

* (MERCURY-192) move virk to common
- also: fix download_file_tree.sh to set permissions on parent dirs

Change-Id: Ic190cb8979055ff4b2d7b0d3a0a14ea35e2b8efa

* (MERCURY-192) add mirror

Change-Id: I52e501420465899dd78fd243a513ed0369909c13

* (MERCURY-192) download soe file and create playbook

Change-Id: I82e460321e1696ed74a277391909920d4b5adaac

* (MERCURY-192) add playbook prep and viya install invocation

Change-Id: Icac0b01932151d5e87899cd645869dcf9a353df7

* (MERCURY-192) move ssl termination
from elb to httpproxy if not cert given.
IAW do not create and delete self signed cert.
The ELB can use TCP/443 instead of HTTPS/443.

Change-Id: I5eed70246d12120d263d925b356d67a88f76e1fa

* (MERCURY-192) add backup vault dir

Change-Id: I2f8df3f4cb137092a6c97d41cea15bea17c53f75

* (MERCURY-192) - virk invocation
- move virk invocation up to the top level (IAAS template)
- need to also move backup vault setup (because it needs the sas user)
  from prepare_nodes to new post_install

Change-Id: I05a0cb7712cfb62eb421187010e929556f02b955

* (MERCURY-192) - implement roles_path lookthrough
- move 'roles' directory out of 'playbooks'
- add 'ansible' directory
- copy host-specific roles into 'ansible/roles'
- also: add log files to cloudwatch config
- move download_file_tree.sh to common

Change-Id: I7707c038fe5a6aade14b040a95f7eaabfc9df4f7

* (MERCURY-192) add early addtl paramater verification
- make ansible controller policies conditional where possible
- add sns messaging

Change-Id: I827309302e27f45dc9a0331f25555cc95478ff64

* (AWSVIYA-103) add sns messages and parm validation
- replace common/download/download_file_tree.sh with faster code
- add pystache file postprocessing
- add send_sns_message.sh script
- remove ssl cert and dns verification (better and faster by CFN)
- add Mirror validation in validate_parameters.sh
- make some ansible controller iam policies conditional
- clean up ansible controller policies syntax

Change-Id: I62fe1516c735b17facadf477039a902c788532f8

* (AWSVIYA-103) add volume tagging
- move mount_disk task from common and add tagging
- template: add permssions for tagging to viya instances role
- clean up permissions syntax

Change-Id: Ia409770dd5afba535380e73fbdb6147f57ae146a

* (AWSVIYA-103) stop/start vm scripts
- move virk under the viya install (required by the virk stop/start playbooks)
- make stop/start scripts parameterless (get all parms from instance metadata)

Change-Id: Ie26833ab9cc0476d0f8bc451400cbe19b2a05af9

* (AWSVIYA-103) update dataagent scriptUse the the more generic version of the dataprep2dataagent playbook
the change in the inventory hosts definition made this necessary.

Change-Id: Id573af863fe582794d341ebebaf6e7488200d486

* (AWSVIYA-43) protect/remove passwords
- some scripts need the admin pw. Set all scripts to 700
- the viya ansible debug log has a dump of sitedefault.yml - remove -v from main playbook invocation;

Change-Id: I4f3e9ce06c8d20c42c46adbebaf24b297fc55a54

* (AWSVIYA-103) adjust recover_cascontroller.sh
- use new install playbooks in script
- replace values in script with pystache
- reenable PlacementGroup
- also: move ViyaVersion from prepare_nodes to prepare_deployment
- also: fix failure message invocation

Change-Id: Ic24dc99c78a508e105c0064370a348d5ccd59217

* (AWSVIYA-103) refactor addaccess.sh script
- also: clean up and comment ansible controller Role policies

Change-Id: Idce452b8186f3c84f83386dbe8e9f0991ea3fb7e

* (AWSVIYA-59) start_viya_vms.sh: stop script on error

Change-Id: I1c7590e75f67c6cb544bae1fc57b0764ad954ff2

* (AWSVIYA-103) add self-signed cert for elb
- add post_deployment/configure_cert role
- add post_deployment plabook
- re-add DelCert lambda function
- add ansible controller policy for iam and qelb actions
- call post_deployment playbook from template

Change-Id: I5590f9d1deeb3e1c82d6e1286841e045739291b2

* (MERCURY-103) clean up file permissions on ansible controller:
- make sure everything is read|write-only for the ec2-user
- remove unused script files
- also: fix cloud-init RC handling for ansible controller

Change-Id: I64484b59759848ab73a5390266c8f67767aad58f

* (AWSVIYA-103) cas_cache permissions for r4
- re-apply ownership/permissions are file mount (this only affects cas cache for r4
the other volumes get their permissions correctly from VIRK)
- make sure to use the ec2-user instead of root (ansible_user_id uses the value of become_user)
- also: add describeTags permission so the volume tagging works (regression instroduced by the permission cleanup in earlier push)

Change-Id: Ie99913b9fc4322affa3e7c55be35d70e74d232d9

* (AWSVIYA-103) fix storage size in topology diagram
- also: lowercase the "i" in "i3" for the cas controller

Change-Id: I81ef645d502f587add13b8262993b9a0625584dd

* (AWSVIYA-103) re-add sns script
turns out we need to keep the sns script because it is called unconditionally,
and the check for operatoremail resp. snstopic is within the script.
That is  actually ok, else we would have checks each time the script is called

Change-Id: I6541b9e370956a7e31db976e2a024d2996561a1f

* (AWSVIYA-112) fix vm restart script
the check for full vm initialization actually fails the script
if the sshd is not yet up (which it is in most cases).
Needed to disable "set -e" for the time of the check.

Change-Id: I40e6381edc81da6005daaf9d564fb082ec14acb1

* (AWSVIYA-63) fix missed host in addldapuser.sh script

Change-Id: I33742cf1282600a6230c2884307e1cf9992be9a8

* point common submodule to github location

* (AWSVIYA-118) move EIP assignment into template

* (AWSVIYA-103) include common code from AWSVIYA-1.4 tag

* (AWSVIYA-125) dataprep utility authentication and error check
use https and -k for sas-admin commands
catch errors in the ansible task;

* (AWSVIYA-103) remove .git directory from common code

* (AWSVIYA-127) mask sasadmin password when passing into prepare_deployment playbook
1 - call prepare_deployment with base64 encoded value for sasadmin pw
2 - generate the recover_cascontroller.sh script with the base64 encoded value
    That script, if run, calls the openldap playbook. It used to get the pw values
    in open code. Now we pass them in already encoded, so there is no need to
    do the encoding in bash

* (AWSVIYA-132) Multiple concurrent deployments intermittently fail

This commit reinstates a "retry" around the site.yml playbook execution in
the 03-install step of the Ansible Controller. When running multiple deployments
of the same software order in parallel, failures occasionally occur when downloading
SAS RPMs via yum.

The retry catches these failures and re-runs the site.yml playbook, up to a maximum of
three attempts.
Latest commit e46dd64 Apr 17, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
ansible
ci
images
scripts
templates
.gitattributes
.gitignore
LICENSE.txt
NOTICE.txt
README.md

README.md

quickstart-sas-viya

SAS Viya on the AWS Cloud

This Quick Start automatically deploys a highly available, production-ready SAS platform on the Amazon Web Services (AWS) Cloud. It deploys SAS Visual Analytics on Linux, SAS Visual Statistics on Linux, or SAS Visual Data Mining and Machine Learning on Linux into a configuration of your choice in about an hour.

This Quick Start uses AWS CloudFormation templates to deploy the SAS Viya products into a virtual private cloud (VPC) in your AWS account. You can build a new VPC for SAS Viya or deploy the software into your existing VPC.

Quick Start SAS Viya Design Architecture

For architectural details, best practices, step-by-step instructions, and customization options, see the deployment guide.

To post feedback, submit feature ideas, or report bugs, use the Issues section of this GitHub repo. If you'd like to submit code for this Quick Start, please review the AWS Quick Start Contributor's Kit.

You can’t perform that action at this time.