© 2019, Cyxtera Cybersecurity, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: (a) redistributions of source code must retain the above copyright notice, this list of conditions and the disclaimer below, and (b) redistributions in binary form must reproduce the above copyright notice, this list of conditions and the disclaimer below in the documentation and/or other materials provided with the distribution. THE CODE AND SCRIPTS POSTED ON THIS WEBSITE ARE PROVIDED ON AN “AS IS” BASIS AND YOUR USE OF SUCH CODE AND/OR SCRIPTS IS AT YOUR OWN RISK. CYXTERA DISCLAIMS ALL EXPRESS AND IMPLIED WARRANTIES, EITHER IN FACT OR BY OPERATION OF LAW, STATUTORY OR OTHERWISE, INCLUDING, BUT NOT LIMITED TO, ALL WARRANTIES OF MERCHANTABILITY, TITLE, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, ACCURACY, COMPLETENESS, COMPATABILITY OF SOFTWARE OR EQUIPMENT OR ANY RESULTS TO BE ACHIEVED THEREFROM. CYXTERA DOES NOT WARRANT THAT SUCH CODE AND/OR SCRIPTS ARE OR WILL BE ERROR-FREE. IN NO EVENT SHALL CYXTERA BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, RELIANCE, EXEMPLARY, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR ANY LOSS OF GOODWILL,LOSS OF ANTICIPATED SAVINGS, COST OF PURCHASING REPLACEMENT SERVICES, LOSS OF PROFITS, REVENUE, DATA OR DATA USE, ARISING IN ANY WAY OUT OF THE USE AND/OR REDISTRIBUTION OF SUCH CODE AND/OR SCRIPTS, REGARDLESS OF THE LEGAL THEORY UNDER WHICH SUCH LIABILITY IS ASSERTED AND REGARDLESS OF WHETHER CYXTERA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH LIABILITY.
Autoscaling AppGate on cloud, supports AWS, Azure and GCP.
AWS provides a feature called autoscaling groups to automatically increase or decrease the amount of instances running based on some metrics such as CPU or memory load.
GCP provides a feature called instance groups to automatically increase or decrease the amount of instances running based on some metrics such as CPU or memory load.
Azure provides a feature called virtual machine scale sets to automatically increase or decrease the amount of instances running based on some metrics such as CPU or memory load.
We can take advantage of that to autoscale the amount of gateways within an AppGate site. We can directly use metrics such as memory and CPU without any specific integration as these are provided by the virtual machine layer.
What remains is registering or deleting these appliances in the controller to make them properly join and leave the AppGate site.
For that we need to put in place two integrations:
- Upscale: Create a new appliance entry and join the AppGate site by seeding that appliance when the autoscaling group spawns a new instance.
- Downscale: Delete the appliance entry when an instance is removed from the autoscaling group.
To automate upscaling we take advantage of the fact that we can pass a script to the appgate instance as userdata for AWS and startup-script for GCP Since we can only pass a single base64 blob as a parameter, the passed script needs to contain all the necessary parameters to create an appliance and export the seed configuration, that is:
- The controller peer hostname or ip.
- The controller CA certificate.
- The admin username and password.
- The site id.
The site id is visible in the URL when navigating to a site on the admin UI. For example, given the site URL https://controller.example.com:444/ui/sites/edit/750f210a-1c42-4d27-b568-4a8767ef2790, the site id is 750f210a-1c42-4d27-b568-4a8767ef2790.
Since we want all our appliances to share a basic template and vary only on parameters such as hostname we need to create a new appliance entry on the controller on our target site. We configure it with all necessary settings except bogus hostnames such as template-gateway.example.com. However, DO NOT activate it. This configuration will be used as a template when creating new instances.
Use the script called appgate-autoscale.py to generate a userdata script with all your parameters. This script requires python3. Then create a autoscaling configurations with all the necessary parameters for a regular AppGate gateway and add the resulting site specific script as userdata on AWS, startup script on GCP and CustomData on Azure.
# Run the bootstrap script and use the resulting script as your launch configuration userdata. python3 appgate-autoscale.py bootstrap --hostname controller.example.com --port 444 --username admin-autoscale-site1 --site 750f210a-1c42-4d27-b568-4a8767ef2790 --cacert mycacert.pem --password myautoscaleadminpassword -f my_example_upscale_script.py
The Python 3 binary is called python.exe, python on MacOS and python3 on linux.
Now when booting up for the first time the instance will find the template appliance configuration as defined on the controller and create a new AppGate appliance entry based on it. It will then export the configuration and seed itself.
By default the upscale script will use the same hostname for the client hostname as for the peer hostname. If you want it to use the client hostname from the template
gateway configuration you can pass the
--share-client-hostname option, for example if you have configured your template with an elastic load balancer hostname.
If a gateway with the same peer hostname already exists in the site, the entry will be deleted by the autoscale script. If the gateway is already activated to the controller the autoscale script will exit and do nothing.
We can use the appgate-autoscale.py script to generate the downscale script as such:
# Run the bootstrap script and use the resulting script as the body of your AWS Lambda function. python3 appgate-autoscale.py bootstrap --action downscale --hostname controller.example.com --port 444 --username admin-autoscale-site1 --site 750f210a-1c42-4d27-b568-4a8767ef2790 --cacert mycacert.pem --password myautoscaleadminpassword -f my_example_downscale_script.py
Whenever an instance is terminated by the autoscaling, this script will run and remove that particular instance from the appliances entry.
To downscale we make use of the appliance shutdown service (cz-shutdown-script service, It is a service that runs the shutdown script at /var/cache/cz-scripts/shutdown-script if it exist) Downscale scrip is automatically generated and place at /var/cache/cz-scripts/shutdown-script by the upscale script.
How to get the controller CA certificate
Go to your controller's admin UI and download the ca certificate in DER form. For example:
Now convert it to PEM form. For example, using openssl:
openssl x509 -in /path/to/AppGate_CA.der -inform DER -outform PEM -out mycacert.pem
Configuring admin privileges for the autoscaling admin user
We will create an admin user with rights limited to autoscaling. This limited admin user can then be used by the autoscaling script.
We will use tags to limit access to only the appliances part of the autoscaling group as well as the template. To do so add a tag on the site's template gateway called siteX-autoscaling-template (It can be named however you want in practice).
The autoscaling script needs rights to view the template appliance for the given site, create a new appliance and export it's seed configuration.
We will create an role called siteX-autoscaling. And add the following privileges:
- View appliances tagged with siteX-autoscaling-template and siteX-autoscaling-instance
- Create appliances with tag siteX-autoscaling-instance
- Export appliances with tag siteX-autoscaling-instance
- Delete appliances with tag siteX-autoscaling-instance (For downscaling)
Now create an admin user called siteX-autoscaling, for example in the local identity provider. Set a strong password, preferably randomly generated. Exempt that user from MFA from the MFA for Admin settings page.
Create a new policy for that user and assign it the siteX-autoscaling role. For the policy assignment add the following two criteria:
- user.ag.identityProviderId is the uuid of the identity provider for the siteX-autoscaling user
- The array tags has item matching siteX-autoscaling
python3 /usr/share/admin-scripts/appgate-autoscale.py upscale controller.example.com --cacert ca-cert.pem --username siteX-autoscaling --password-path password-executable --site 750f210a-1c42-4d27-b568-4a8767ef2790 > /home/cz/seed.json