New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Azure - Container host docs #4732
Merged
stefangordon
merged 8 commits into
cloud-custodian:master
from
axis7818:container-host-docs
Sep 9, 2019
Merged
Changes from 5 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
848ffd4
more prereq info
axis7818 1197f07
initial rework of container host docs
axis7818 0d76d49
create the ACI tutorial
axis7818 6f11880
improvements to docs
axis7818 3534325
better docs for policy modes
axis7818 df4b432
diagram for multiple hosts
axis7818 219ad04
hosting options note
axis7818 666f88c
Merge branch 'master' into container-host-docs
stefangordon File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
.. _azure_configuration_acitutorial: | ||
|
||
Tutorial - ACI Deployment | ||
========================= | ||
|
||
This is a step-by-step tutorial for deploying the Azure Container Host in ACI that runs custodian | ||
policies that are uploaded to an Azure Storage Account. This tutorial makes use of the | ||
`Azure CLI <https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest>`_, | ||
so make sure that it is installed and you are logged in to your subscription. | ||
|
||
1. Create a Resource Group | ||
-------------------------- | ||
|
||
This will hold all of the Azure resources created in this tutorial. You could use an existing | ||
resource group too. | ||
|
||
.. code-block:: bash | ||
|
||
az group create --name c7n-aci-tutorial --location westus2 | ||
|
||
|
||
2. Create a Storage Account | ||
--------------------------- | ||
|
||
This storage account will hold all of the blobs and queues used by the container host. You could | ||
also use an existing resource here or multiple storage accounts. | ||
|
||
We will also create a blob container. One will host the uploaded policies, and the other will store | ||
the custodian output files of running policies. | ||
|
||
.. code-block:: bash | ||
|
||
az storage account create --resource-group c7n-aci-tutorial --name c7nstorage | ||
|
||
account_key=$(az storage account keys list --account-name c7nstorage --query "[0].value" --output tsv) | ||
az storage container create --account-name c7nstorage --account-key $account_key --name c7n-aci-policies | ||
|
||
3. Create a Managed Identity | ||
---------------------------- | ||
|
||
This identity will be used with MSI on the ACI instance. You could also use an existing identity. | ||
|
||
.. code-block:: bash | ||
|
||
az identity create --resource-group c7n-aci-tutorial --name c7n-aci | ||
|
||
This identity will need the proper permissions to interact with the storage account and any other | ||
azure resources that the Azure Container Host interacts with. This includes the resources that | ||
policies will run against. | ||
|
||
The simplest way to grant these permissions is to make the identity a ``Contributor`` on the target | ||
subscription and a ``Storage Blob Data Contributor`` and ``Storage Queue Data Contributor`` on the | ||
storage account created above. | ||
|
||
.. code-block:: bash | ||
|
||
az role assignment create --assignee <Managed Identity Client Id> \ | ||
--role "Contributor" --scope <Target Subscription Resource Id> | ||
|
||
az role assignment create --assignee <Managed Identity Client Id> \ | ||
--role "Storage Blob Data Contributor" --scope <Storage Account Resource Id> | ||
|
||
az role assignment create --assignee <Managed Identity Client Id> \ | ||
--role "Storage Queue Data Contributor" --scope <Storage Account Resource Id> | ||
|
||
4. Create the ACI Container Host | ||
-------------------------------- | ||
|
||
Now all of the required Azure resources should exist, and we can deploy the Container Host ARM template. | ||
Create a parameters file called ``c7n-aci-tutorial.json`` | ||
|
||
.. code-block:: json | ||
|
||
{ | ||
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", | ||
"contentVersion": "1.0.0.0", | ||
"parameters": { | ||
"aci_name": { | ||
"value": "cloud-custodian" | ||
}, | ||
"user_assigned_identity_name": { | ||
"value": "c7n-aci" | ||
}, | ||
"azure_subscription_id": { | ||
"value": "<Target Subscription Id>" | ||
}, | ||
"azure_event_queue_name": { | ||
"value": "c7n-aci-events" | ||
}, | ||
"azure_container_storage": { | ||
"value": "https://c7nstorage.blob.core.windows.net/c7n-aci-policies" | ||
}, | ||
"azure_event_queue_resource_id": { | ||
"value": "<Storage Account Resource Id>" | ||
} | ||
} | ||
} | ||
|
||
And deploy with this command | ||
|
||
.. code-block:: bash | ||
|
||
az group deployment create \ | ||
--resource-group c7n-aci-tutorial \ | ||
--template-file tools/ops/azure/container-host/aci/aci-template.json \ | ||
--parameters @c7n-aci-tutorial.json | ||
|
||
Once this deployment succeeds, the Azure Container Host should be running! You can see the logs | ||
with the following command: | ||
|
||
.. code-block:: bash | ||
|
||
az container logs --resource-group c7n-aci-tutorial --name cloud-custodian --follow | ||
|
||
5. Upload a Custodian Policy | ||
---------------------------- | ||
|
||
Finally, create a custodian policy called ``find-c7nstorage.yaml``. This policy will just find the | ||
storage account we made earlier. We'll set the mode to run every minute for easier testing. | ||
|
||
.. code-block:: yaml | ||
|
||
policies: | ||
- name: find-c7nstorage | ||
resource: azure.storage | ||
mode: | ||
type: container-periodic | ||
schedule: "* * * * *" # Run every minute as an example | ||
filters: | ||
- type: value | ||
key: name | ||
op: eq | ||
value: c7nstorage | ||
|
||
|
||
Upload this file to the policy storage container. Within a few minutes, the container host should | ||
pick it up and begin executing it. | ||
|
||
.. code-block:: bash | ||
|
||
az storage blob upload --account-name c7nstorage --account-key $account_key \ | ||
--container-name c7n-aci-policies --file find-c7nstorage.yaml --name find-c7nstorage.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,174 +3,112 @@ | |
Azure Container Hosting | ||
======================= | ||
|
||
The Azure provider can be run in a containerized environment using the official cloud custodian | ||
`docker image <https://hub.docker.com/r/cloudcustodian/c7n>`_. The Azure Container Host knows | ||
how to handle both periodic and event based policies. | ||
The Azure Container Host is an alternate execution mode for the cloud custodian azure provider. | ||
Running the Azure Container Host is done with the `official custodian docker image <https://hub.docker.com/r/cloudcustodian/c7n>`_. | ||
See the :ref:`ACI <azure_configuration_acitutorial>` and :ref:`Kubernetes <azure_configuration_helmtutorial>` deployment tutorials | ||
to get started running the Azure Container Host. | ||
|
||
The Azure Container Host will read policies from an Azure Blob Container and, depending on their mode, | ||
will listen for event triggers or run them periodically. Event triggers are pulled from an event | ||
queue within a storage account. | ||
How It Works | ||
############ | ||
|
||
To run the container, pass the docker command as `/usr/local/bin/python3 -m c7n_azure.container_host.host` | ||
and authenticate through environment variables (see :ref:`azure_authentication`). It is also important | ||
to make sure that the container host is authenticated as a contributor on the policy storage and a | ||
message processor on the event queue. | ||
The Azure Container Host will periodically scan azure blob storage for a set of custodian policies | ||
to execute in either a periodic or event based mode against a target subscription. For periodic | ||
policies, the container host will execute the policy on the cron schedule that is provided. For event | ||
based policies, the container host maintains an azure queue that subscribes to events in the target | ||
azure subscription. | ||
|
||
There are 3 important environment variables that are specific to the container host. | ||
Once the Azure Container Host is deployed, any policies uploaded to blob storage are automatically | ||
loaded and running against an Azure Subscription. This makes it very easy to manage and run a large | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
number of policies. | ||
|
||
* `AZURE_EVENT_QUEUE_RESOURCE_ID`: The resource ID for a storage account for the event queue. | ||
* `AZURE_EVENT_QUEUE_NAME`: The name of the event queue. If this queue does not exist, it will be created. | ||
* `AZURE_CONTAINER_STORAGE`: The URI to an azure blob container that will hold all of the policies for this container host. | ||
.. image:: resources/container-host.png | ||
axis7818 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Supported Policy Modes | ||
###################### | ||
|
||
The container host will only run policies with one of the following modes specified. Otherwise, | ||
the policy will be ignored. | ||
|
||
- container-periodic | ||
Periodic | ||
^^^^^^^^ | ||
|
||
Run the policy periodically based on a provided crontab schedule. | ||
|
||
.. c7n-schema:: mode.container-periodic | ||
|
||
- container-event | ||
|
||
Run the policy when particular events are dropped into the specified event queue. | ||
|
||
.. c7n-schema:: mode.container-event | ||
|
||
Deployment Options | ||
################## | ||
|
||
Azure Container Instance | ||
------------------------ | ||
|
||
The ARM template to deploy the Azure Container Host is provided for deploying an ACI instance | ||
against a single subscription using a `user assigned identity <https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview>`_ | ||
for authentication. | ||
|
||
Here is an example deployment of the ARM template using the azure cli: | ||
|
||
.. code-block:: bash | ||
|
||
az group deployment create \ | ||
--resource-group my-resource-group \ | ||
--template-file tools/ops/azure/container-host/aci/aci-template.json \ | ||
--parameters \ | ||
aci_name=cloud-custodian \ | ||
user_assigned_identity_name=my-uai \ | ||
azure_subscription_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ | ||
azure_event_queue_name=custodian-aci-queue \ | ||
azure_container_storage=https://myStorageAccount.blob.core.windows.net/aci-policies \ | ||
azure_event_queue_resource_id=/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/my-resource-group/providers/Microsoft.Storage/storageAccounts/myStorageAccount | ||
|
||
Kubernetes (Helm Chart) | ||
----------------------- | ||
|
||
A helm chart is provided that will deploy a set of cloud custodian containers against a set of | ||
subscriptions to be monitored. For information on how to customize the values, reference | ||
the helm chart's values.yaml. | ||
Periodic policies must specify a mode with type ``container-periodic`` and a cron schedule. This | ||
schedule can specify when the policy should run. For example: once every hour, on midnight on every | ||
weekday, or once a month. | ||
|
||
.. code-block:: yaml | ||
|
||
# sample-values.yaml | ||
policies: | ||
- name: run-every-day-at-midnight | ||
resource: azure.resourcegroup | ||
mode: | ||
type: container-periodic | ||
schedule: '0 0 * * *' | ||
|
||
defaultEnvironment: | ||
AZURE_TENANT_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" | ||
AZURE_CLIENT_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" | ||
AZURE_EVENT_QUEUE_NAME: "cloud-custodian-events" | ||
|
||
defaultSecretEnvironment: | ||
AZURE_CLIENT_SECRET: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" | ||
.. c7n-schema:: mode.container-periodic | ||
|
||
subscriptionHosts: | ||
- name: "my-first-subscription" | ||
environment: | ||
AZURE_SUBSCRIPTION_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" | ||
AZURE_CONTAINER_STORAGE: "https://firstStorageAccount.blob.core.windows.net/cloud-custodian-policies" | ||
AZURE_EVENT_QUEUE_RESOURCE_ID: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myResourceGroup/providers/Microsoft.Storage/storageAccounts/firstStorageAccount" | ||
- name: "my-second-subscription" | ||
environment: | ||
AZURE_SUBSCRIPTION_ID: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" | ||
AZURE_CONTAINER_STORAGE: "https://secondStorageAccount.blob.core.windows.net/more-policies" | ||
AZURE_EVENT_QUEUE_RESOURCE_ID: "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myOtherResourceGroup/providers/Microsoft.Storage/storageAccounts/secondStorageAccount" | ||
Event Based | ||
^^^^^^^^^^^ | ||
|
||
To deploy the chart: | ||
Event based policies must specify a mode with the type ``container-event`` and a set of events that | ||
will trigger the execution. For example: after a new resource group is created. | ||
|
||
.. code-block:: bash | ||
|
||
helm upgrade --install --debug --namespace cloud-custodian --values /path/to/sample-values.yaml my-cloud-custodian-deployment tools/ops/azure/container-host/chart | ||
|
||
|
||
Helm Chart Deployment Script | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
Additionally, a utility script for deploying the helm chart against either a single subscription | ||
or all subscriptions in a management group is provided. When deploying for a management group, | ||
all of the containers will share the same policy storage and storage account for event queues. | ||
|
||
.. code-block:: bash | ||
|
||
# Usage | ||
$ python tools/ops/azure/container-host/chart/deploy_chart.py --help | ||
|
||
Usage: deploy_chart.py [OPTIONS] COMMAND [ARGS]... | ||
|
||
Options: | ||
-d, --deployment-name TEXT | ||
-n, --deployment-namespace TEXT | ||
-v, --helm-values-file TEXT [required] | ||
-s, --helm-set TEXT | ||
--dry-run / --no-dry-run | ||
--help Show this message and exit. | ||
|
||
Commands: | ||
management_group | ||
subscription | ||
|
||
|
||
# subscription subcommand | ||
$ python tools/ops/azure/container-host/chart/deploy_chart.py subscription --help | ||
.. code-block:: yaml | ||
|
||
Usage: deploy_chart.py subscription [OPTIONS] | ||
policies: | ||
- name: run-on-new-resource-group | ||
resource: azure.resourcegroup | ||
mode: | ||
type: container-event | ||
events: | ||
- resourceProvider: Microsoft.Resources/subscriptions/resourceGroups | ||
event: write | ||
|
||
Options: | ||
-i, --subscription-id TEXT [required] | ||
--help Show this message and exit. | ||
.. c7n-schema:: mode.container-event | ||
|
||
Configuration | ||
############# | ||
|
||
# management_group subcommand | ||
$ python tools/ops/azure/container-host/chart/deploy_chart.py management_group --help | ||
Configuration for the container host is provided as environment variables. | ||
There are several environment variables specific to the container host: | ||
axis7818 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Usage: deploy_chart.py management_group [OPTIONS] | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| Variable Name | Required | Description | | ||
+===================================+==========+=====================================================================================================================================================================+ | ||
| ``AZURE_CONTAINER_STORAGE`` | required | The URL to the azure blob container to load custodian policies from. | | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| ``AZURE_EVENT_QUEUE_RESOURCE_ID`` | required | The resource id of the storage account to hold the event queue. | | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| ``AZURE_EVENT_QUEUE_NAME`` | required | The name of the event queue that the container host will listen on. If this does not exist, it will be created. | | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| ``AZURE_CONTAINER_LOG_GROUP`` | | The application insights to send log output to. In the format: ``azure://<instrumentation_key_guid>``. | | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| ``AZURE_CONTAINER_METRICS`` | | The application insights to send metrics output to. In the format: ``azure://<instrumentation_key_guid>``. | | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| ``AZURE_CONTAINER_OUTPUT_DIR`` | | The URL of the storage account blob container to send log output to. In the format: ``azure://<storage_account_name>.blob.core.windows.net/<blob_container_name>``. | | ||
+-----------------------------------+----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
|
||
Options: | ||
-i, --management-group-id TEXT [required] | ||
--help Show this message and exit. | ||
In additiona to the above environment variables, authentication must be provided to the container host. | ||
See :ref:`azure_authentication` for authenticating the container host with an azure identity. | ||
|
||
Examples | ||
________ | ||
Once an identity has been established, it will need the following roles in azure: | ||
|
||
Deploy against a single subscription: | ||
- ``Reader`` and ``Storage Blob Data Contributor`` on the Storage Account that holds the policy files. | ||
|
||
.. code-block:: bash | ||
- ``Contributor`` and ``Storage Queue Message Processor`` on the Storage Account that the event queue will live in. | ||
|
||
python cloud-custodian/tools/ops/azure/container-host/chart/deploy_chart.py \ | ||
--deployment-name azure-c7n \ | ||
--deployment-namespace cloud-custodian \ | ||
--helm-values-file path/to/my-values.yaml \ | ||
--helm-set defaultSecretEnvironment.AZURE_CLIENT_SECRET=some-secret-value \ | ||
subscription --subscription-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx | ||
- Any other roles that are needed to run the policies that the container host will run. For example, if there is a policy that filters the ``azure.vm`` resource, the ``Reader`` role will be required for the VMs that are in the container host's target subscription. | ||
|
||
Deploy against a management group: | ||
Running Locally | ||
############### | ||
|
||
.. code-block:: bash | ||
The container host can be run locally with ``python -m c7n_azure.container_host.host``. | ||
You will need to provide all of the same configuration specified above through either environment | ||
variables or CLI options. Run ``python -m c7n_azure.container_host.host --help`` for more information. | ||
|
||
python tools/ops/azure/container-host/chart/deploy_chart.py \ | ||
--deployment-name azure-c7n \ | ||
--deployment-namespace cloud-custodian \ | ||
--helm-values-file path/to/my-values.yaml \ | ||
--helm-set defaultSecretEnvironment.AZURE_CLIENT_SECRET=some-secret-value \ | ||
management_group --management-group-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx | ||
Deployment Options | ||
################## | ||
|
||
For quick deployments, we provide tooling for 2 methods of deploying the Azure Container Host: | ||
:ref:`ACI <azure_configuration_acitutorial>`, and | ||
:ref:`Kubernetes with a Helm chart <azure_configuration_helmtutorial>`. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reads very well. I will go through it step by step tomorrow to make sure that each step works