AzureRM delegated user access?

[hashibot]

This issue was originally opened by @glenjamin as hashicorp/terraform#12208. It was migrated here as part of the provider split. The original body of the issue is below.

---

Terraform Version

terraform 0.8.7

Affected Resource(s)

Azure Provider

I've been using terraform to configure a few things with the azurerm provider, which is broadly working well.

The part I'm not entirely keen on is how I have to create an Azure AD application that has appropriate permissions to perform resource operations per user who I want to have that ability - and annoyingly that azure apps can't be added to groups for permissions.

Clearly this isn't really terraform's fault. Azure AD applications are also allowed to run in a delegated resource mode, where they perform commands as the user in question - this is how the NodeJS, Python CLI apps and the Powershell Cmdlets work. The python code samples are relatively easy to follow here: https://github.com/AzureAD/azure-activedirectory-library-for-python

Currently terraform is effectively using the "Acquire Token with Client Credentials" method whereas the other CLI tools use "Acquire Token with device code".

Is this something that terraform could be extended to support?

Annoyingly there's no existing go code in the AzureAD org, but the pieces it uses all seem to be standard parts of OAuth2. The other downside is that the token only lasts for 1 hour, but does provide a refresh token that can be used to get a fresh one.

Perhaps an "easy" extension would be to allow terraform to accept the fully resolved azure bearer token, and then it would be possible for users to use their own mechanism to get and keep a valid bearer token if they wanted?

[codyja]

This would be great for us also. We can't really use Terraform in Azure because all the devs would need Service Principal accounts, but they're not really tied to the person's user account. So if they leave, the SPN would still be active even if the user account is disabled.

Ansible currently allows the user to specify their ad_user, password, and subscription when making API calls. I don't think this is the "Aquire Token with device code" method.

It looks like they are using azure.common.credentials UserPassCredentials if this helps at all.

Here's more information on the different methods in the azure SDK.
https://docs.microsoft.com/en-us/python/azure/python-sdk-azure-authenticate?view=azure-python

[colemickens]

There's already support in the Go SDK (I authored it), this should be easy to support.

@tombuildsstuff What's the precident for this though? As is, the creds are effectively part of the stored state. The auth context with device login is an ephemeral access_token (maybe there's a refresh too, but if so, it's only good for a certain window of time).

Does it break the Terraform model to have the authentication context of the deployment be excluded from the tfstate?

[nbering]

@colemickens I don't see credentials anywhere in my state file, and Terraform has no issue if I change credentials to another AD Service Principal against the same state. On our team each user has a different AD application to represent them, so I don't think a change in token with each session will be an issue.

I also don't think it's practical to use the Device Token flow unless the user provides an existing access token. There just doesn't seem to be a good place to put that type of auth workflow into the terraform execution flow, considering that the provider is just a plugin for a larger application with different flow concerns.

The other way would be to have the user provide their password to Terraform. This is not ideal from a security practices perspective. But it will end up being the path of least resistance if User authentication is desirable over use of a Service Principle. This is what @codyja was referencing from the python SDK and Ansible.

While I personally discourage the handling of passwords by a third-party application like this, in the end all you can really do is make users aware of what the best practice is, and let them decide. Using User credentials because it helps manage the threat model for a whole organization is a perfectly valid point of view that pokes a little hole in my view that the Service Principle was "the one right way" for this type of tooling support scenario.

[NoseyNick]

Providing a password to Terraform is a bad idea, and also unlikely to work in many cases where multiple SAML referrals and MFA and things are going on. I think the only realistic option is to check ~/.azure/accessTokens.json and use those. If they have expired, call "az login" or "azure login", OR re-implement the same "visit https://aka.ms/devicelogin and enter the code ABC123456" functionality, and WRITE to ~/.azure/accessTokens.json (so we can mix 'n' match use of CLI commands and Terraform)

[nbering]

Implementing the actual device token flow seems way outside the norm for a terraform provider plugin. Reading from the ~/.azure/accessTokens.json seems reasonable, if somewhat complicated for people with multiple subscriptions and Azure AD tenants. Perhaps the file can be checked after looking for an environment variable, to be in line with AWS checking the shared credentials file.

[codyja]

What both @NoseyNick and @nbering are mentioning sound like pure gold. Just some way to authenticate with the user account (like authenticating with powershell or azure cli). We also have MFA on our subscriptions/tenant.

We currently authenticate against Terraform against AWS, by using a custom python package we found on github to generate the MFA auth against Okta (MFA) and AWS. It presents short lived (1 hr) token/keys to ~/.aws/credentials for the desired subscription. Terraform then checks for ENV variables, or the ~/.aws/credentials file for keys. https://github.com/rracterr/okta_aws_login

[colemickens]

re-implement the same "visit https://aka.ms/devicelogin and enter the code ABC123456" functionality, and WRITE to ~/.azure/accessTokens.json (so we can mix 'n' match use of CLI commands and Terraform)

That is two function calls from the Go SDK (so, fairly easy to implement), though @nbering seems to be pushing back. If Terraform reads the accessToken from the CLI, that's probably fine, but I'm not sure it would be a "good citizen" thing to start writing into the CLI's token cache.

The CLI also has a method exposed now that should return a fresh token. (aka, it will refresh if necessary, or prompt the user for re-auth). So presumably Terraform could just os.Exec and then read the token from std in.

It sounds like the preference would be for the latter approach? If so, I can dig up the relevant PR to the azure-cli and rope in their folks to weigh in.

[PeterWeiler]

Having Terraform use ~/.azure/accessTokens.json to get the Azure access token would be great, and is consistent with how Terraform can currently use ~/.aws/credentials to get AWS access token. How to populate those token/credential files is the user's job, and can be done via the AWS/Azure CLI tools.

[colemickens]

@PeterWeiler Is there a significant opinion about reaching into the CLI's private token cache directly, versus going through a supported public API (aka, os.Exec of az get-token (or whatever the exact command is)?

[nbering]

Correct me if I'm wrong, but currently there are no providers that actually provide an interactive auth experience. Terraform will prompt for missing variables or provider configuration values, but providers exist in a separate process and I am not aware of a way for - as an example - the plugin to test a token and then request credentials if the token is expired. The only thing the provider can really do is fail with a meaningful error message so that the user can replace their token and try again. Terraform core just checks for the presence of the token, not the validity.

Just to play devil's advocate here - as I often do - what would the expected behaviour be if a Terraform Apply operation were sufficiently complex that it could not be completed in the lifetime of one token? What does it do now? Would the provider support refresh tokens? I've never had anything run that long, but I've seen it come close while waiting for a name to become available after deleting and recreating a storage account.

[colemickens]

To clarify slightly, what I was referencing was a command in az which outputs an access token, but is also non-interactive. I'm suggesting that's a better choice than directly reaching into the token cache (undocumented/unsupported as far as I know), and achieves the same goals as achieved with AWS (non-interactive, doesn't require out-of-band credential/privileges provisioning).

This allows az to manage refreshing, etc as is necessary.

cc: @kris-nova

[codyja]

This sounds very reasonable to me.

[tombuildsstuff]

@colemickens I've spent a little while looking into reusing the Azure CLI tokens and this appears to be a viable route forward - as such I'm working through the dependencies needed to make this possible (read: a Go-AutoRest PR + updating/removing Riviera). Once that's done we can continue with this :)

Thanks!

[NoseyNick]

"az account get-access-token" seems to be the CLI2.0 command, which spits out a JSON object with the token+details for one subscription. I'm not aware of an "azure" CLI1.0 / xplat version of this command though, but they DO both share the same ~/.azure/accessTokens.json, which seems to contain a list of the tokens for ALL your subscriptions, in exactly the same format.

Personally I'd probably prefer to see them coming from ~/.azure/accessTokens.json - which I'd argue is almost as well documented as "az account get-access-token" anyway... but EITHER approach is better than SP-logins-only which we're stuck with right now.

[kongou-ae]

If terraform can use ~/.azure/accessTokens.json, using terraform becomes very easy on Azure Cloud Shell. Because when Azure Cloud Shell started, it already has ~/.azure/accessTokens.json. It's not necessary to create service principal.

[asbjorn-wiik]

A powershell script using this function https://gallery.technet.microsoft.com/scriptcenter/Easily-obtain-AccessToken-3ba6e593/ that enumerate an environment variable for Terraform to consume. This would also simplify things for Terraform to Azure users that have MFA enabled accounts. So when can we start using user access tokens with Terraform? Next week would be great! Thanks :D

[asbjorn-wiik]

Lifetime of access tokens is now configurable in Azure to a maximum of 1 day per this documentation https://docs.microsoft.com/en-us/azure/active-directory/active-directory-configurable-token-lifetimes/

[tombuildsstuff]

👋 @colemickens @codyja @nbering @NoseyNick @asbjorn-wiik @kongou-ae

I spent some time looking into this last week and have a prototype running locally which supports reading the accessToken.json and azureProfiles.json files to obtain the access tokens from the Azure CLI. I've sent PR's to the Azure/go-autorest repository to expose these fields so that we can use them, which has been merged.

Whilst working on and testing this, I've stumbled upon a few serialization bugs in the Azure CLI which only occur in certain environments, which will need to be fixed before we can proceed (or we'll have bug reports from these users). To fix this I've reached out to the relevant teams at Microsoft to get this fixed, and I'm hopeful we have a road forward in the near future - but we're really blocked from proceeding until this is fixed unfortunately.

I'll post when I've got an update - but this is certainly something we're looking into :)

Thanks!

[asbjorn-wiik]

@colemickens @codyja @nbering @NoseyNick @kongou-ae @tombuildsstuff
I assume that Terraform is using the Azure REST API when working with Azure and that the terraform-provider-azurerm is using subscription_id, client_id, client_secret and tenant_id to get an access token for further interaction with Azures REST API.
Terraform already retrieves all those values for environment variables ARM_SUBSCRIPTION_ID, ARM_CLIENT_ID, ARM_CLIENT_SECRET and ARM_TENANT_ID. If you can try retrieving access token from environment variables.
Then here is a very simple powershell script that sets a ARM_SUBSCRIPTION_ID, ARM_TENANT_ID and ARM_ACCESSTOKEN console environment variable that Terraform could consume.
Get-TerraformConsumableAzureAccessToken.ps1.txt

[colemickens]

Hm, I don't want to be too stubborn and I'll leave it be after this, but I want to again suggest that a supported API is used instead of reaching into the json files directly:

1. It will actually be supported instead of relying on a convention (though I admit, it's probably relatively stable)
2. It allows Azure CLI to handle the lifecycle of the token. I don't think Terraform or go-autorest should try to handle token refreshing and management. Maybe it's not so bad though?
3. It would presumably avoid the serialization bugs (I assume get-access-token parses and then reserializes the output).

[colemickens]

I'm guessing @tombuildsstuff doesn't plan to reach into the Windows Credential Store to retrieve the access token that PS caches, so that PowerShell script will probably be very useful for Windows users. Might be a new feature request to respect ARM_ACCESS_TOKEN though (I like the idea!)

[colemickens]

Whilst I'd agree with you here - this approach was recommended by the Azure CLI team (for both this, and taking into account some future requirements), which is why we took this direction (it's also consistent with how we handle this in AWS). Happy to revisit in the future however :)

Good enough for me! :) Thanks @tombuildsstuff, you rock!

(Also, IIRC and assuming they don't change things, you can use the refresh token "independently" without affecting the ability of the CLI to later use an older refresh token. [whether or not that's a good thing, I'm not sure]).

Thanks again, very sincerely!

[asbjorn-wiik]

Hi, again
Don’t want to come off as pushy but, is there an ETA for when utilization of user access token will be solved for AzureRM?
The reason I’m asking is that I work for service provider that is a AWS Global Channel Reseller and a Microsoft Tier 1 Cloud Solution Provider, we also help customers with their journey to public cloud if the customer so wishes.
We started with AWS and are using Terraform for creating AWS cloud solutions. Terraform working in conjunction AWS we have a decent and secure access management and pick up an audit of whom did what in AWS accounts. Terraform with Azure and service principals is a security abyss and therefore minefield form where ye shall not pass. Visual Studio Team Service gets away with using service principals because VSTS in of itself has access and audit to “delicate parts” such as service principals.
Azure already has access management and audit in place and we don’t want to establish another one with VSTS if we can avoid it and we also like the synergies that a same tooling scenario gives us.
So, it’s decision making time for me and I need some information, hope you have something concrete.

Thanks

[tombuildsstuff]

Hey @asbjorn-wiik

Don’t want to come off as pushy but, is there an ETA for when utilization of user access token will be solved for AzureRM?

I've opened PR #316 which has initial support for this - but it's dependent on some upstream changes being merged to be able to proceed. As such we're unable to commit to a date for this beyond "soon" at this point in time - but we'll post an update as soon as we can :)

Thanks!

[tombuildsstuff]

👋 everybody

The upstream changes have been released with Azure/go-autorest v8.4 - and have been incorporated in #316 - which has been merged. This means support for logging in with the Azure CLI will be shipping in v0.2.0 of the AzureRM provider, which I'm hoping to release later today :)

Thanks!

[codyja]

@tombuildsstuff Wow! We look forward to it!

[tombuildsstuff]

👋 again everybody

Just to let you know that we've just released v0.2.0 of the AzureRM Provider which includes support for authentication via the Azure CLI. There's some documentation available here to explain what's needed to set this up on Azure CLI front - and we'd love to hear your feedback :)

Thanks!

cc @codyja @colemickens @asbjorn-wiik @kongou-ae @NoseyNick @nbering @PeterWeiler @glenjamin

[kongou-ae]

Great!!! I checked that terraform runs with only 'provider "azurerm" {}'.

[glenjamin]

I just went to go and test this, but it looks like it only applies to the azure provider - not the azure backend. 😞

Was this distinction intentional?

[tombuildsstuff]

Hey @glenjamin

it looks like it only applies to the azure provider - not the azure backend

I'll try and take a look into adding support for this at some point this week - however given the Backends are released with Terraform Core we're bound to that release schedule.

Was this distinction intentional?

It wasn't (thanks for letting us know!) - the Provider (which supports AzureCLI auth) is from Terraform Core (which is where the Backend lives, which currently doesn't support Azure CLI auth). As such it wasn't intentionally left out (however I'd not considered it) - as mentioned above we'll try and take a look into this this week and I'll re-open this issue in the interim until this is supported.

Thanks!

[asbjorn-wiik]

Hi @tombuildsstuff.

First of all great work so fare.
I hope in the future you'll support logged in user Powershell session access token.

Do you know if this feature also will be implemented into Packer?

Thanks.

Best regards Asbjørn Wiik

[tombuildsstuff]

Hey @asbjorn-wiik

I hope in the future you'll support logged in user Powershell session access token.

I might be wrong, but haven't the PowerShell Cmdlets been deprecated in favour of Azure CLI 2.0?

Do you know if this feature also will be implemented into Packer?

I believe a similar feature is present in Packer's Azure support - but the best place to ask would be on the Packer repository :)

Thanks!

[asbjorn-wiik]

Hi @tombuildsstuff

You might be referring to 'Azure ActiveDirectory (MSOnline)' Powershell module that is been announced being deprecated in favor of 'Azure Active Directory version 2' Powershell module.
Documented here https://docs.microsoft.com/en-us/powershell/azure/active-directory/install-msonlinev1?view=azureadps-2.0/

Powershell is Microsofts holy child so they will provide it with all the toys(modules) it could ever need for all of the foreseeable future :-)

I'll scoot over to the Packer repo and try to find a suitable issue.

Thanks.

[yooakim]

Great work @tombuildsstuff ! While I normally use Service Principals ith Azure it is great to be able to run Terraform with my user credentials!

It would be nice if the module could warn/inform users who try this from the PowerShell command line that it requires AZ CLI 2.x. and does not work with PowerShell (no rush to implement PowerShell as long as it is clear to anyone who tries to use it that it has to be via AZ CLI 2.x).

[glenjamin]

I've just been skimming the PR to see how this will work, and there's one quirk I wasn't expecting.

It looks as though the subscription ID that's used will always be the one marked as default in the logged-in azure profile - even if a different subscription ID is referenced in the provider block.

In my current project we deal with multiple subscriptions, and various credentials to access them - being able to specify the subscription in each sub-folder works a nice a check that you're using the right credentials when running terraform.

If I'm reading correctly, if you had a different default subscription selected than had been used with the current tfstate, you'd get a terraform plan which intended to create every resource again, which could lead to confusion.

Would it be possible to match the subscription ID against the one in provider config if present, and only use the default profile subscription when it is omitted?

[cschyma]

Unfortunately the new Azure RM provider did not solve the issue for me:

az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code DUYF5NRD5 to authenticate.
[
  {
    "cloudName": "AzureCloud",
    "id": "<subscriptionId>",
    "isDefault": true,
    "name": "Visual Studio Enterprise",
    "state": "Enabled",
    "tenantId": "<tenantId>",
    "user": {
      "name": "<user>",
      "type": "user"
    }
  }
]

The Azure login was successful, but I still get this error message:

terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------
Error running plan: 1 error(s) occurred:

* provider.azurerm: Azure CLI Authorization Profile was not found. Please ensure the Azure CLI is installed and then log-in with `az login`.

The Terraform file looks like the following:

provider "azurerm" {
   version = "~> 0.2"
}

resource "azurerm_resource_group" "example" {
  name     = "example"
  location = "westeurope"
}

[timbuchwaldt]

As raised in #432 the provider doesn't support changing the subscription and tenant, although this is documented as options to the provider.

Is there any chance to get this behaviour? I am building a deployment that should establish a VPN connection between two tenants and therefore need two different tenant IDs to be used.

[glenjamin]

@tombuildsstuff Sorry to ping directly, but did a related issue on core get opened for this?

[tombuildsstuff]

@glenjamin thanks for creating a Core issue for this. Rather than tracking the pending half of this request on a separate repository (CLI Auth for Backends) - I'm going to suggest we close this issue and track the outstanding work in hashicorp/terraform#16763

Thanks!

[pricoppd]

Unfortunately the new Azure RM provider did not solve the issue for me:

az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code DUYF5NRD5 to authenticate.
[
  {
    "cloudName": "AzureCloud",
    "id": "<subscriptionId>",
    "isDefault": true,
    "name": "Visual Studio Enterprise",
    "state": "Enabled",
    "tenantId": "<tenantId>",
    "user": {
      "name": "<user>",
      "type": "user"
    }
  }
]

The Azure login was successful, but I still get this error message:

terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------
Error running plan: 1 error(s) occurred:

* provider.azurerm: Azure CLI Authorization Profile was not found. Please ensure the Azure CLI is installed and then log-in with `az login`.

The Terraform file looks like the following:

provider "azurerm" {
   version = "~> 0.2"
}

resource "azurerm_resource_group" "example" {
  name     = "example"
  location = "westeurope"
}

[pricoppd]

@cschyma, have you managed to find a fix for this?

[tombuildsstuff]

@pricoppd the version of the AzureRM Provider you're using is very old (~> 0.2) - we're currently on v1.21 - I'd suggest upgrading to the latest release and trying again :)

Since this issue has been resolved (and so that we don't notify all the other folks on the thread) I'm going to lock this issue for the moment - but please feel free to open a new issue if you're still having issues and we'll take a look

Thanks!