Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure - Container host docs #4732

Merged
merged 8 commits into from Sep 9, 2019
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
142 changes: 142 additions & 0 deletions docs/source/azure/configuration/acitutorial.rst
@@ -0,0 +1,142 @@
.. _azure_configuration_acitutorial:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads very well. I will go through it step by step tomorrow to make sure that each step works


Tutorial - ACI Deployment
=========================

This is a step-by-step tutorial for deploying the Azure Container Host in ACI that runs custodian
policies that are uploaded to an Azure Storage Account. This tutorial makes use of the
`Azure CLI <https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest>`_,
so make sure that it is installed and you are logged in to your subscription.

1. Create a Resource Group
--------------------------

This will hold all of the Azure resources created in this tutorial. You could use an existing
resource group too.

.. code-block:: bash

az group create --name c7n-aci-tutorial --location westus2


2. Create a Storage Account
---------------------------

This storage account will hold all of the blobs and queues used by the container host. You could
also use an existing resource here or multiple storage accounts.

We will also create a blob container. One will host the uploaded policies, and the other will store
the custodian output files of running policies.

.. code-block:: bash

az storage account create --resource-group c7n-aci-tutorial --name c7nstorage

account_key=$(az storage account keys list --account-name c7nstorage --query "[0].value" --output tsv)
az storage container create --account-name c7nstorage --account-key $account_key --name c7n-aci-policies

3. Create a Managed Identity
----------------------------

This identity will be used with MSI on the ACI instance. You could also use an existing identity.

.. code-block:: bash

az identity create --resource-group c7n-aci-tutorial --name c7n-aci

This identity will need the proper permissions to interact with the storage account and any other
azure resources that the Azure Container Host interacts with. This includes the resources that
policies will run against.

The simplest way to grant these permissions is to make the identity a ``Contributor`` on the target
subscription and a ``Storage Blob Data Contributor`` and ``Storage Queue Data Contributor`` on the
storage account created above.

.. code-block:: bash

az role assignment create --assignee <Managed Identity Client Id> \
--role "Contributor" --scope <Target Subscription Resource Id>

az role assignment create --assignee <Managed Identity Client Id> \
--role "Storage Blob Data Contributor" --scope <Storage Account Resource Id>

az role assignment create --assignee <Managed Identity Client Id> \
--role "Storage Queue Data Contributor" --scope <Storage Account Resource Id>

4. Create the ACI Container Host
--------------------------------

Now all of the required Azure resources should exist, and we can deploy the Container Host ARM template.
Create a parameters file called ``c7n-aci-tutorial.json``

.. code-block:: json

{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"aci_name": {
"value": "cloud-custodian"
},
"user_assigned_identity_name": {
"value": "c7n-aci"
},
"azure_subscription_id": {
"value": "<Target Subscription Id>"
},
"azure_event_queue_name": {
"value": "c7n-aci-events"
},
"azure_container_storage": {
"value": "https://c7nstorage.blob.core.windows.net/c7n-aci-policies"
},
"azure_event_queue_resource_id": {
"value": "<Storage Account Resource Id>"
}
}
}

And deploy with this command

.. code-block:: bash

az group deployment create \
--resource-group c7n-aci-tutorial \
--template-file tools/ops/azure/container-host/aci/aci-template.json \
--parameters @c7n-aci-tutorial.json

Once this deployment succeeds, the Azure Container Host should be running! You can see the logs
with the following command:

.. code-block:: bash

az container logs --resource-group c7n-aci-tutorial --name cloud-custodian --follow

5. Upload a Custodian Policy
----------------------------

Finally, create a custodian policy called ``find-c7nstorage.yaml``. This policy will just find the
storage account we made earlier. We'll set the mode to run every minute for easier testing.

.. code-block:: yaml

policies:
- name: find-c7nstorage
resource: azure.storage
mode:
type: container-periodic
schedule: "* * * * *" # Run every minute as an example
filters:
- type: value
key: name
op: eq
value: c7nstorage


Upload this file to the policy storage container. Within a few minutes, the container host should
pick it up and begin executing it.

.. code-block:: bash

az storage blob upload --account-name c7nstorage --account-key $account_key \
--container-name c7n-aci-policies --file find-c7nstorage.yaml --name find-c7nstorage.yaml
214 changes: 76 additions & 138 deletions docs/source/azure/configuration/containerhosting.rst
Expand Up @@ -3,174 +3,112 @@
Azure Container Hosting
=======================

The Azure provider can be run in a containerized environment using the official cloud custodian
`docker image <https://hub.docker.com/r/cloudcustodian/c7n>`_. The Azure Container Host knows
how to handle both periodic and event based policies.
The Azure Container Host is an alternate execution mode for the cloud custodian azure provider.
Running the Azure Container Host is done with the `official custodian docker image <https://hub.docker.com/r/cloudcustodian/c7n>`_.
See the :ref:`ACI <azure_configuration_acitutorial>` and :ref:`Kubernetes <azure_configuration_helmtutorial>` deployment tutorials
to get started running the Azure Container Host.

The Azure Container Host will read policies from an Azure Blob Container and, depending on their mode,
will listen for event triggers or run them periodically. Event triggers are pulled from an event
queue within a storage account.
How It Works
############

To run the container, pass the docker command as `/usr/local/bin/python3 -m c7n_azure.container_host.host`
and authenticate through environment variables (see :ref:`azure_authentication`). It is also important
to make sure that the container host is authenticated as a contributor on the policy storage and a
message processor on the event queue.
The Azure Container Host will periodically scan azure blob storage for a set of custodian policies
to execute in either a periodic or event based mode against a target subscription. For periodic
policies, the container host will execute the policy on the cron schedule that is provided. For event
based policies, the container host maintains an azure queue that subscribes to events in the target
azure subscription.

There are 3 important environment variables that are specific to the container host.
Once the Azure Container Host is deployed, any policies uploaded to blob storage are automatically
loaded and running against an Azure Subscription. This makes it very easy to manage and run a large
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loaded and run against maybe?

number of policies.

* `AZURE_EVENT_QUEUE_RESOURCE_ID`: The resource ID for a storage account for the event queue.
* `AZURE_EVENT_QUEUE_NAME`: The name of the event queue. If this queue does not exist, it will be created.
* `AZURE_CONTAINER_STORAGE`: The URI to an azure blob container that will hold all of the policies for this container host.
.. image:: resources/container-host.png
axis7818 marked this conversation as resolved.
Show resolved Hide resolved

Supported Policy Modes
######################

The container host will only run policies with one of the following modes specified. Otherwise,
the policy will be ignored.

- container-periodic
Periodic
^^^^^^^^

Run the policy periodically based on a provided crontab schedule.

.. c7n-schema:: mode.container-periodic

- container-event

Run the policy when particular events are dropped into the specified event queue.

.. c7n-schema:: mode.container-event

Deployment Options
##################

Azure Container Instance
------------------------

The ARM template to deploy the Azure Container Host is provided for deploying an ACI instance
against a single subscription using a `user assigned identity <https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview>`_
for authentication.

Here is an example deployment of the ARM template using the azure cli:

.. code-block:: bash

az group deployment create \
--resource-group my-resource-group \
--template-file tools/ops/azure/container-host/aci/aci-template.json \
--parameters \
aci_name=cloud-custodian \
user_assigned_identity_name=my-uai \
azure_subscription_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
azure_event_queue_name=custodian-aci-queue \
azure_container_storage=https://myStorageAccount.blob.core.windows.net/aci-policies \
azure_event_queue_resource_id=/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/my-resource-group/providers/Microsoft.Storage/storageAccounts/myStorageAccount

Kubernetes (Helm Chart)
-----------------------

A helm chart is provided that will deploy a set of cloud custodian containers against a set of
subscriptions to be monitored. For information on how to customize the values, reference
the helm chart's values.yaml.
Periodic policies must specify a mode with type ``container-periodic`` and a cron schedule. This
schedule can specify when the policy should run. For example: once every hour, on midnight on every
weekday, or once a month.

.. code-block:: yaml

# sample-values.yaml
policies:
- name: run-every-day-at-midnight
resource: azure.resourcegroup
mode:
type: container-periodic
schedule: '0 0 * * *'

defaultEnvironment:
AZURE_TENANT_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
AZURE_CLIENT_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
AZURE_EVENT_QUEUE_NAME: "cloud-custodian-events"

defaultSecretEnvironment:
AZURE_CLIENT_SECRET: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
.. c7n-schema:: mode.container-periodic

subscriptionHosts:
- name: "my-first-subscription"
environment:
AZURE_SUBSCRIPTION_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
AZURE_CONTAINER_STORAGE: "https://firstStorageAccount.blob.core.windows.net/cloud-custodian-policies"
AZURE_EVENT_QUEUE_RESOURCE_ID: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myResourceGroup/providers/Microsoft.Storage/storageAccounts/firstStorageAccount"
- name: "my-second-subscription"
environment:
AZURE_SUBSCRIPTION_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
AZURE_CONTAINER_STORAGE: "https://secondStorageAccount.blob.core.windows.net/more-policies"
AZURE_EVENT_QUEUE_RESOURCE_ID: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myOtherResourceGroup/providers/Microsoft.Storage/storageAccounts/secondStorageAccount"
Event Based
^^^^^^^^^^^

To deploy the chart:
Event based policies must specify a mode with the type ``container-event`` and a set of events that
will trigger the execution. For example: after a new resource group is created.

.. code-block:: bash

helm upgrade --install --debug --namespace cloud-custodian --values /path/to/sample-values.yaml my-cloud-custodian-deployment tools/ops/azure/container-host/chart


Helm Chart Deployment Script
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Additionally, a utility script for deploying the helm chart against either a single subscription
or all subscriptions in a management group is provided. When deploying for a management group,
all of the containers will share the same policy storage and storage account for event queues.

.. code-block:: bash

# Usage
$ python tools/ops/azure/container-host/chart/deploy_chart.py --help

Usage: deploy_chart.py [OPTIONS] COMMAND [ARGS]...

Options:
-d, --deployment-name TEXT
-n, --deployment-namespace TEXT
-v, --helm-values-file TEXT [required]
-s, --helm-set TEXT
--dry-run / --no-dry-run
--help Show this message and exit.

Commands:
management_group
subscription


# subscription subcommand
$ python tools/ops/azure/container-host/chart/deploy_chart.py subscription --help
.. code-block:: yaml

Usage: deploy_chart.py subscription [OPTIONS]
policies:
- name: run-on-new-resource-group
resource: azure.resourcegroup
mode:
type: container-event
events:
- resourceProvider: Microsoft.Resources/subscriptions/resourceGroups
event: write

Options:
-i, --subscription-id TEXT [required]
--help Show this message and exit.
.. c7n-schema:: mode.container-event

Configuration
#############

# management_group subcommand
$ python tools/ops/azure/container-host/chart/deploy_chart.py management_group --help
Configuration for the container host is provided as environment variables.
There are several environment variables specific to the container host:
axis7818 marked this conversation as resolved.
Show resolved Hide resolved

Usage: deploy_chart.py management_group [OPTIONS]
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Variable Name | Required | Description |
+===================================+==========+=====================================================================================================================================================================+
| ``AZURE_CONTAINER_STORAGE`` | required | The URL to the azure blob container to load custodian policies from. |
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``AZURE_EVENT_QUEUE_RESOURCE_ID`` | required | The resource id of the storage account to hold the event queue. |
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``AZURE_EVENT_QUEUE_NAME`` | required | The name of the event queue that the container host will listen on. If this does not exist, it will be created. |
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``AZURE_CONTAINER_LOG_GROUP`` | | The application insights to send log output to. In the format: ``azure://<instrumentation_key_guid>``. |
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``AZURE_CONTAINER_METRICS`` | | The application insights to send metrics output to. In the format: ``azure://<instrumentation_key_guid>``. |
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ``AZURE_CONTAINER_OUTPUT_DIR`` | | The URL of the storage account blob container to send log output to. In the format: ``azure://<storage_account_name>.blob.core.windows.net/<blob_container_name>``. |
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Options:
-i, --management-group-id TEXT [required]
--help Show this message and exit.
In additiona to the above environment variables, authentication must be provided to the container host.
See :ref:`azure_authentication` for authenticating the container host with an azure identity.

Examples
________
Once an identity has been established, it will need the following roles in azure:

Deploy against a single subscription:
- ``Reader`` and ``Storage Blob Data Contributor`` on the Storage Account that holds the policy files.

.. code-block:: bash
- ``Contributor`` and ``Storage Queue Message Processor`` on the Storage Account that the event queue will live in.

python cloud-custodian/tools/ops/azure/container-host/chart/deploy_chart.py \
--deployment-name azure-c7n \
--deployment-namespace cloud-custodian \
--helm-values-file path/to/my-values.yaml \
--helm-set defaultSecretEnvironment.AZURE_CLIENT_SECRET=some-secret-value \
subscription --subscription-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
- Any other roles that are needed to run the policies that the container host will run. For example, if there is a policy that filters the ``azure.vm`` resource, the ``Reader`` role will be required for the VMs that are in the container host's target subscription.

Deploy against a management group:
Running Locally
###############

.. code-block:: bash
The container host can be run locally with ``python -m c7n_azure.container_host.host``.
You will need to provide all of the same configuration specified above through either environment
variables or CLI options. Run ``python -m c7n_azure.container_host.host --help`` for more information.

python tools/ops/azure/container-host/chart/deploy_chart.py \
--deployment-name azure-c7n \
--deployment-namespace cloud-custodian \
--helm-values-file path/to/my-values.yaml \
--helm-set defaultSecretEnvironment.AZURE_CLIENT_SECRET=some-secret-value \
management_group --management-group-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Deployment Options
##################

For quick deployments, we provide tooling for 2 methods of deploying the Azure Container Host:
:ref:`ACI <azure_configuration_acitutorial>`, and
:ref:`Kubernetes with a Helm chart <azure_configuration_helmtutorial>`.