title | description | services | author | ms.topic | ms.date | ms.author | ms.reviewer | ms.service | ms.custom |
---|---|---|---|---|---|---|---|---|---|
Integration of virtual network injection with Chaos Studio |
Learn how to use virtual network injection with Azure Chaos Studio. |
chaos-studio |
prashabora |
conceptual |
10/26/2022 |
abbyweisberg |
nikhilkaul |
chaos-studio |
devx-track-azurecli |
Azure Virtual Network is the fundamental building block for your private network in Azure. A virtual network enables many types of Azure resources to securely communicate with each other, the internet, and on-premises networks. A virtual network is similar to a traditional network that you operate in your own datacenter. It brings other benefits of Azure's infrastructure, such as scale, availability, and isolation.
Virtual network injection allows an Azure Chaos Studio resource provider to inject containerized workloads into your virtual network so that resources without public endpoints can be accessed via a private IP address on the virtual network. After you've configured virtual network injection for a resource in a virtual network and enabled the resource as a target, you can use it in multiple experiments. An experiment can target a mix of private and nonprivate resources if the private resources are configured according to the instructions in this article.
We are also now excited to share that Chaos Studio supports running agent-based experiments using Private Endpoints! Chaos Studio now supports Private Link for both service-direct and agent-based experiments. If you would like to use Private-Link for agent-based experiments, please reach out to your CSA or visit How to: Setup private link for agent-based experiments. For private link for service-direct faults, read the following sections for instructions on how to use them.
Currently, you can only enable certain resource types for Chaos Studio virtual network injection:
- Azure Kubernetes Service (AKS) targets can be enabled with virtual network injection through the Azure portal and the Azure CLI. All AKS Chaos Mesh faults can be used.
- Azure Key Vault targets can be enabled with virtual network injection through the Azure CLI. The faults that can be used with virtual network injection are Disable Certificate, Increment Certificate Version, and Update Certificate Policy.
To use Chaos Studio with virtual network injection, you must meet the following requirements.
- The
Microsoft.ContainerInstance
andMicrosoft.Relay
resource providers must be registered with your subscription. - The virtual network where Chaos Studio resources will be injected must have two subnets: a container subnet and a relay subnet. A container subnet is used for the Chaos Studio containers that will be injected into your private network. A relay subnet is used to forward communication from Chaos Studio to the containers inside the private network.
- Both subnets need at least
/28
for the size of the address space (in this case/27
is larger than/28
, for example). An example is an address prefix of10.0.0.0/28
or10.0.0.0/24
. - The container subnet must be delegated to
Microsoft.ContainerInstance/containerGroups
. - The subnets can be arbitrarily named, but we recommend
ChaosStudioContainerSubnet
andChaosStudioRelaySubnet
.
- Both subnets need at least
- When you enable the desired resource as a target so that you can use it in Chaos Studio experiments, the following properties must be set:
- Set
properties.subnets.containerSubnetId
to the ID for the container subnet. - Set
properties.subnets.relaySubnetId
to the ID for the relay subnet.
- Set
If you're using the Azure portal to enable a private resource as a Chaos Studio target, Chaos Studio currently only recognizes subnets named ChaosStudioContainerSubnet
and ChaosStudioRelaySubnet
. If these subnets don't exist, the portal workflow can create them automatically.
If you're using the CLI, the container and relay subnets can have any name (subject to the resource naming guidelines). Specify the appropriate IDs when you enable the resource as a target.
This example shows how to configure a private AKS cluster to use with Chaos Studio. It assumes you already have a private AKS cluster within your Azure subscription. To create one, see Create a private Azure Kubernetes Service cluster.
-
In the Azure portal, go to Subscriptions > Resource providers in your subscription.
-
Register the
Microsoft.ContainerInstance
andMicrosoft.Relay
resource providers, if they aren't already registered, by selecting the provider and then selecting Register. Reregister theMicrosoft.Chaos
resource provider.:::image type="content" source="images/vnet-register-resource-provider.png" alt-text="Screenshot that shows how to register a resource provider." lightbox="images/vnet-register-resource-provider.png":::
-
Go to Chaos Studio and select Targets. Find your desired AKS cluster and select Enable targets > Enable service-direct targets.
:::image type="content" source="images/vnet-enable-targets.png" alt-text="Screenshot that shows how to enable targets in Chaos Studio." lightbox="images/vnet-enable-targets.png":::
-
Select the cluster's virtual network. If the virtual network already includes subnets named
ChaosStudioContainerSubnet
andChaosStudioRelaySubnet
, select them. If they don't already exist, they're automatically created for you.:::image type="content" source="images/vnet-select-subnets.png" alt-text="Screenshot that shows how to select the virtual network and subnets." lightbox="images/vnet-select-subnets.png":::
-
Select Review + Enable > Enable.
:::image type="content" source="images/vnet-review.png" alt-text="Screenshot that shows how to review the target enablement." lightbox="images/vnet-review.png":::
Now you can use your private AKS cluster with Chaos Studio. To learn how to install Chaos Mesh and run the experiment, see Create a chaos experiment that uses a Chaos Mesh fault with the Azure portal.
-
Register the
Microsoft.ContainerInstance
andMicrosoft.Relay
resource providers with your subscription by running the following commands. If they're both already registered, you can skip this step. For more information, see the Register resource provider instructions.az provider register --namespace 'Microsoft.ContainerInstance' --wait
az provider register --namespace 'Microsoft.Relay' --wait
Verify the registration by running the following commands:
az provider show --namespace 'Microsoft.ContainerInstance' | grep registrationState
az provider show --namespace 'Microsoft.Relay' | grep registrationState
In the output, you should see something similar to:
"registrationState": "Registered",
-
Reregister the
Microsoft.Chaos
resource provider with your subscription.az provider register --namespace 'Microsoft.Chaos' --wait
Verify the registration by running the following command:
az provider show --namespace 'Microsoft.Chaos' | grep registrationState
In the output, you should see something similar to:
"registrationState": "Registered",
-
Create two subnets in the virtual network you want to inject Chaos Studio resources into (in this case, the private AKS cluster's virtual network):
- Container subnet (example name:
ChaosStudioContainerSubnet
)- Delegate the subnet to the
Microsoft.ContainerInstance/containerGroups
service. - This subnet must have at least
/28
in the address space.
- Delegate the subnet to the
- Relay subnet (example name:
ChaosStudioRelaySubnet
)- This subnet must have at least
/28
in the address space.
- This subnet must have at least
az network vnet subnet create -g MyResourceGroup --vnet-name MyVnetName --name ChaosStudioContainerSubnet --address-prefixes "10.0.0.0/28" --delegations "Microsoft.ContainerInstance/containerGroups"
az network vnet subnet create -g MyResourceGroup --vnet-name MyVnetName --name ChaosStudioRelaySubnet --address-prefixes "10.0.0.0/28"
- Container subnet (example name:
-
When you enable targets for the AKS cluster so that you can use it in chaos experiments, set the
properties.subnets.containerSubnetId
andproperties.subnets.relaySubnetId
properties by using the new subnets you created in step 3.Replace
$SUBSCRIPTION_ID
with your Azure subscription ID. Replace$RESOURCE_GROUP
and$AKS_CLUSTER
with the resource group name and your AKS cluster resource name. Also, replace$AKS_INFRA_RESOURCE_GROUP
and$AKS_VNET
with your AKS infrastructure resource group name and virtual network name. Replace$URL
with the correspondinghttps://management.azure.com/
URL used for onboarding the target.CONTAINER_SUBNET_ID=/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$AKS_INFRA_RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/$AKS_VNET/subnets/ChaosStudioContainerSubnet RELAY_SUBNET_ID=/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$AKS_INFRA_RESOURCE_GROUP/providers/Microsoft.Network/virtualNetworks/$AKS_VNET/subnets/ChaosStudioRelaySubnet BODY="{ \"properties\": { \"subnets\": { \"containerSubnetId\": \"$CONTAINER_SUBNET_ID\", \"relaySubnetId\": \"$RELAY_SUBNET_ID\" } } }" az rest --method put --url $URL --body "$BODY"
Now you can use your private AKS cluster with Chaos Studio. To learn how to install Chaos Mesh and run the experiment, see Create a chaos experiment that uses a Chaos Mesh fault with the Azure CLI.
- Virtual network injection is currently only possible in subscriptions/regions where Azure Container Instances and Azure Relay are available.
- When you create a Target resource that you enable with virtual network injection, you need
Microsoft.Network/virtualNetworks/subnets/write
access to the virtual network. For example, if the AKS cluster is deployed to virtual network_A, then you must have permissions to create subnets in virtual network_A to enable virtual network injection for the AKS cluster.
Now that you understand how virtual network injection can be achieved for Chaos Studio, you're ready to: