-
Notifications
You must be signed in to change notification settings - Fork 21.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement AAD authentication for Azure Storage #38803
Conversation
@blob_service = Azure::Storage::Blob::BlobService.create(storage_account_name: storage_account_name, storage_access_key: storage_access_key, **options) | ||
@shared_access_signature = Azure::Storage::Common::Core::Auth::SharedAccessSignature.new(storage_account_name, storage_access_key) | ||
end | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I’ll have more feedback on these polymorphic clients, but for starters, let’s break them into separate files:
lib/active_storage/service/azure_storage_service/active_directory_client.rb
(and remove Azure from the class name; the namespace makes it clear that it applies to Azure)lib/active_storage/service/azure_storage_service/access_key_client.rb
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in a689cbb
# container: "" | ||
# tenant_id: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa | ||
# client_id: bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb | ||
# client_secret: cccccccc-cccc-cccc-cccc-cccccccccccc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any pointers on setting this up for CI? I’m pretty unfamiliar with Azure in general.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you already have a storage account you're using on CI? If so, all we need to do is create a service principal and grant it the appropriate IAM permissions to the storage account.
You can do this via a snippet similar to the one I posted in the PR description :
service_principal_name="enter some name here"
storage_account_name="enter your storage account name here"
container_name="enter the name for the container used for the new tests here"
# fetch the resource id of the existing storage account
storage_account_id="$(az storage account show --name "${storage_account_name}" --query id --output tsv)"
# create the container that the new tests will use
# we need to do this as a one-time setup step since for extra security we'll only grant
# our service principal access to this specific container instead of the entire storage account
# which means that the service principal won't be able to dynamically create new containers
az storage container create --name "${container_name}" --account-name "${storage_account_name}"
# create the service principal we'll be using for the tests
sp="$(az ad sp create-for-rbac --name "${service_principal_name}" --skip-assignment --output json)"
sp_tenant="$(jq -r '.tenant' <<< "${sp}")"
sp_client="$(jq -r '.appId' <<< "${sp}")"
sp_secret="$(jq -r '.password' <<< "${sp}")"
# grant the service principal the appropriate IAM permissions on the storage account
# the data contributor role is required for read/write/list/delete operations on blobs
# in the container and the delegator permission is required to create signed private urls
az role assignment create --role "Storage Blob Data Contributor" --assignee "${sp_client}" --scope "${storage_account_id}/blobServices/default/containers/${container_name}"
az role assignment create --role "Storage Blob Delegator" --assignee "${sp_client}" --scope "${storage_account_id}"
# last but not least, use these values to update configurations.yml
cat << EOM
azure_aad:
service: AzureStorage
storage_account_name: "${storage_account_name}"
tenant_id: "${sp_tenant}"
client_id: "${sp_client}"
client_secret: "${sp_secret}"
container: "${container_name}"
EOM
Alternatively you can also do this in the Azure Portal: see the docs for creating a service principal and the docs for assigning a storage role.
If you do not already have a storage account that's being used in the CI or wish to create a new one, you can use the snippet that I posted in the pull request description to create all the required resources and update the configurations.yml
file as required.
Hope this helps. Let me know what additional assistance I can provide.
Currently, the Azure implementation of ActiveStorage supports shared key authentication [1]. This commit extends the functionality to also support authentication via Azure Active Directory (AAD) [2]. Authenticating to Azure Storage via AAD enables using role-based access control (RBAC) to manage access to the storage resources. This enables multi-tenancy use-cases, such as for example having multiple Rails applications use different containers in the same Azure Storage account while ensuring that each application can only access data in its own container. To enable AAD RBAC authentication, configure the tenant ID, client ID and client secret of the AAD principal that should be used for authentication: ```yaml azure: service: AzureStorage storage_account_name: mystorageaccount container: mycontainer tenant_id: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa client_id: bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb client_secret: <%= Rails.application.credentials.dig(:azure, :client_secret) %> ``` Note that the AAD principal used for authentication must be granted at least the roles "Storage Blob Delegator" on the storage account as well as "Storage Blob Data Contributor" on the container. The former role is required for generating shared access signatures for direct uploads and asset URLs and the latter role is required for read/write/delete/list permissions on the contents of the container. Also note that the AAD authentication flow relies on user delegation keys [3] which were introduced in Azure Storage API version 2018-11-09. This means that as of update 1811 the functionality will not yet work when targeting Azure Stack Hub Storage. For testing purposes, all the required resources and permissions to exercise the AAD authentication flow can be set up with the following script: ```bash service_principal_name="rails-azure-aad-integration-testes" # CHANGE ME location="eastus" # CHANGE ME subscription="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" # CHANGE ME storage_account_name="railsazureaad" # CHANGE ME container_name="integrationtests" # CHANGE ME az login az account set -s "${subscription}" az group create -l "${location}" -n "${resource_group_name}" az storage account create -l "${location}" -g "${resource_group_name}" -n "${storage_account_name}" --kind StorageV2 az storage container create -n "${container_name}" --connection-string "$(az storage account show-connection-string -n "${storage_account_name}")" sp="$(az ad sp create-for-rbac --name "${service_principal_name}" --skip-assignment)" sp_tenant="$(jq -r '.tenant' <<< "${sp}")" sp_client="$(jq -r '.appId' <<< "${sp}")" sp_secret="$(jq -r '.password' <<< "${sp}")" while ! az role assignment create --role "Storage Blob Data Contributor" --assignee "${sp_client}" --scope "/subscriptions/${subscription}/resourceGroups/${resource_group_name}/providers/Microsoft.Storage/storageAccounts/${storage_account_name}/blobServices/default/containers/${container_name}"; do sleep 2s; done while ! az role assignment create --role "Storage Blob Delegator" --assignee "${sp_client}" --scope "/subscriptions/${subscription}/resourceGroups/${resource_group_name}/providers/Microsoft.Storage/storageAccounts/${storage_account_name}"; do sleep 2s; done cat >> activestorage/test/service/configurations.yml << EOF azure_aad: service: AzureStorage storage_account_name: "${storage_account_name}" tenant_id: "${sp_tenant}" client_id: "${sp_client}" client_secret: "${sp_secret}" container: "${container_name}" EOF ``` [1] https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key [2] https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-azure-active-directory [3] https://docs.microsoft.com/en-us/rest/api/storageservices/get-user-delegation-key
@georgeclaghorn Could you take another look at this pull request? Are there any additional questions or concerns I can help address and/or pointers to provide for setting up the integration tests in CI? I'm not 100% certain, but the latest test failures don't seem related to this pull request (and indeed at one point when I originally opened the pull request I recall that the build passed). Could you clarify this and/or provide some pointers for me to debug the failures? Thanks in advance! |
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
Summary
Currently, the Azure implementation of ActiveStorage supports shared key authentication. This commit extends the functionality to also support authentication via Azure Active Directory (AAD).
Other Information
Authenticating to Azure Storage via AAD enables using role-based access control (RBAC) to manage access to the storage resources. This enables multi-tenancy use-cases, such as for example having
multiple Rails applications use different containers in the same Azure Storage account while ensuring that each application can only access data in its own container.
To enable AAD RBAC authentication, configure the tenant ID, client ID and client secret of the AAD principal that should be used for authentication:
Note that the AAD principal used for authentication must be granted at least the roles "Storage Blob Delegator" on the storage account as well as "Storage Blob Data Contributor" on the container. The former role is required for generating shared access signatures for direct uploads and asset URLs and the latter role is required for read/write/delete/list permissions on the contents of the container.
Also note that the AAD authentication flow relies on user delegation keys which were introduced in Azure Storage API version 2018-11-09. This means that as of update 1811 the functionality will not yet work when targeting Azure Stack Hub Storage.
For testing purposes, all the required resources and permissions to exercise the AAD authentication flow can be set up with the following script (assuming the Azure CLI and JQ are installed):
Validated by @michaelperel
CC @jrodbeta