`azurerm_storage_account`: Support managed HSMs for CMKs #25088

Botje · 2024-02-28T21:10:36Z

This PR adds separate arguments to the customer_managed_key block to supply HSM-backed CMKs.
For now it just adds a counterpart to the key vault uri / key ID, which does mean that the storage_account_customer_managed_key resource accepts a managed hsm uri + key name + version whereas storage_account accepts a HSM key ID.
Future work could add the HSM key ID to the storage_account_customer_managed_key resource as well.

It does not address the problem raised in this comment by @tombuildsstuff but tries to use the suffixes as published in the environment.

resolves #23321, resolves #14389

katbyte · 2024-02-28T23:52:45Z

I think this may be related to this pr? #25069

Botje · 2024-02-29T08:11:24Z

Thanks for that pointer. Yes, the ManagedHSMKeyID struct and associated parse/validate methods would be very helpful here. Do you suggest I wait until that gets merged and adapt to the new types and methods, or can this be merged separately and can I submit a cleanup PR later? Or I could cherry-pick these changes and hope there are no revisions needed...

Botje · 2024-03-01T13:22:02Z

I rebased my changes on top of (a part of) the mentioned PR.

riemers · 2024-03-06T10:20:56Z

Would this not also apply to say keyvault and disk encryption sets as you experience the same issue there too?

Botje · 2024-03-06T10:35:27Z

Absolutely! There is a comment in the source code to the effect that the various implementations of customer_managed_key blocks ought to be centralized so the same code only has to be written once. I have an approach in mind but would prefer to do it after HSM support is added.

riemers · 2024-03-12T12:34:46Z

From what i heard, once the #25069 is merged, it should be based of that. As if you now plan/apply on a resource that already has a hsm key it will fail to read the key out properly because of the nestedid's being used.

Botje · 2024-03-12T13:08:22Z

@riemers I am not sure I follow. This PR is partially based on the mentioned PR, see "wuxu92 and others added 5 commits" in the commit list.

As my PR stands currently, the read path for storage_account_customer_managed_key just copies the URI to the managed_hsm_uri attribute and the validation for that is (much too lax, on review) "is it a HTTPS URL".

The read path for storage_account uses an HSM-specific "nested item ID", which really should be renamed to "HSM key id".

If you have spotted something I missed do please tell me via code comment or a simple test case, so I can fix it.

riemers · 2024-03-12T15:47:53Z

I overlooked the part where you said "redid the work based on the PR" Just wanted to know if that other PR is merged, if that would fix the Error: internal-error: Managed HSM IDs are not supported as Key Vault Nested Item if the resource gets read out. Since you can still use the azapi provider until the other items are properly supported.

Botje · 2024-03-12T20:37:20Z

#25069 will not fix the issue you mention by itself, because it does not change the function used to process the IDs in the various customer_managed_key blocks that my PR touches.

riemers · 2024-03-13T08:14:33Z

Thanks for the info, suppose we just have to wait a bit more until parts are moved forward.

tombuildsstuff · 2024-03-22T10:57:43Z

Just to aid internal triage (since this otherwise appears ready for review) - since this PR's dependent on a couple of other PRs I'm going to mark this PR as Blocked temporarily until the dependent PR's are addressed - this isn't indicative of an issue in this PR, it's purely to help from an ordering/triage perspective for the moment.

Botje · 2024-03-22T11:04:58Z

Understood, thanks. I will keep track of #25069 and #25324 and rework when needed.

…ane Versioned and Versionless Key IDs

riemers · 2024-04-24T06:40:51Z

FYI, there is a internal ticket under IPL-6341 with Hashicorp to look at it. All we can do now is wait i suppose ;)

katbyte · 2024-05-02T16:28:00Z

#25601 has been merged which has the updated id parsers and validators and unblocks this pr

err != nil -> err == nil

mbfrahry

LGTM

katbyte

LGTM 🏗️

<Actions> <action id="f410411e63aff4bb73a81c2aec1d373cf8a903e63b30dee2006b0030d8a94cc8"> <h3>Bump Terraform `azurerm` provider version</h3> <details id="1d9343c012f5434ac9fe8a98135bae3667b399259be16d9b14302ea3bd424a24"> <summary>Update Terraform lock file</summary> <p>changes detected:
	"hashicorp/azurerm" updated from "3.101.0" to "3.102.0" in file ".terraform.lock.hcl"</p> <details> <summary>3.102.0</summary> <pre>Changelog retrieved from:
	https://github.com/hashicorp/terraform-provider-azurerm/releases/tag/v3.102.0
FEATURES:

* New Resource: `azurerm_storage_sync_server_endpoint` ([#25831](https://github.com/hashicorp/terraform-provider-azurerm/issues/25831))
* New Resource: `azurerm_storage_container_immutability_policy` ([#25804](https://github.com/hashicorp/terraform-provider-azurerm/issues/25804))

ENHANCEMENTS:

* `azurerm_load_test` - add support for `encryption` ([#25759](https://github.com/hashicorp/terraform-provider-azurerm/issues/25759))
* `azurerm_network_connection_monitor` - update validation for `target_resource_type` and `target_resource_id` ([#25745](https://github.com/hashicorp/terraform-provider-azurerm/issues/25745))
* `azurerm_mssql_managed_database` - support for a Restorable Database ID to be used as the `source_database_id` for point in time restore ([#25568](https://github.com/hashicorp/terraform-provider-azurerm/issues/25568))
* `azurerm_storage_account` - support for the `managed_hsm_key_id` property ([#25088](https://github.com/hashicorp/terraform-provider-azurerm/issues/25088))
* `azurerm_storage_account_customer_managed_key` - support for the `managed_hsm_key_id` property ([#25088](https://github.com/hashicorp/terraform-provider-azurerm/issues/25088))

BUG FIXES:

* `azurerm_linux_function_app` - now sets docker registry url in `linux_fx_version` by default ([#23911](https://github.com/hashicorp/terraform-provider-azurerm/issues/23911))
* `azurerm_resource_group` - work around sporadic eventual consistency errors ([#25758](https://github.com/hashicorp/terraform-provider-azurerm/issues/25758))

DEPRECATIONS:

* `azurerm_key_vault_managed_hardware_security_module_role_assignment` - the `vault_base_url` property has been deprecated in favour of the `managed_hsm_id` property ([#25601](https://github.com/hashicorp/terraform-provider-azurerm/issues/25601))


</pre> </details> </details> <a href="https://infra.ci.jenkins.io/job/updatecli/job/azure/job/main/148/">Jenkins pipeline link</a> </action> </Actions> --- <table> <tr> <td width="77"> <img src="https://www.updatecli.io/images/updatecli.png" alt="Updatecli logo" width="50" height="50"> </td> <td> <p> Created automatically by <a href="https://www.updatecli.io/">Updatecli</a> </p> <details><summary>Options:</summary> <br /> <p>Most of Updatecli configuration is done via <a href="https://www.updatecli.io/docs/prologue/quick-start/">its manifest(s)</a>.</p> <ul> <li>If you close this pull request, Updatecli will automatically reopen it, the next time it runs.</li> <li>If you close this pull request and delete the base branch, Updatecli will automatically recreate it, erasing all previous commits made.</li> </ul> <p> Feel free to report any issues at <a href="https://github.com/updatecli/updatecli/issues">github.com/updatecli/updatecli</a>.<br /> If you find this tool useful, do not hesitate to star <a href="https://github.com/updatecli/updatecli/stargazers">our GitHub repository</a> as a sign of appreciation, and/or to tell us directly on our <a href="https://matrix.to/#/#Updatecli_community:gitter.im">chat</a>! </p> </details> </td> </tr> </table> Co-authored-by: Jenkins Infra Bot (updatecli) <60776566+jenkins-infra-bot@users.noreply.github.com>

github-actions bot added documentation service/storage service/managed-hsm size/L waiting-response and removed waiting-response labels Feb 28, 2024

Botje force-pushed the storage-account-cmk branch from 604c342 to 6bb1a73 Compare February 28, 2024 22:04

github-actions bot removed the waiting-response label Feb 28, 2024

Botje force-pushed the storage-account-cmk branch from 6bb1a73 to 47db274 Compare March 1, 2024 13:14

github-actions bot added size/XL waiting-response and removed size/L labels Mar 1, 2024

Botje force-pushed the storage-account-cmk branch from 47db274 to e993524 Compare March 1, 2024 13:20

github-actions bot removed the waiting-response label Mar 1, 2024

tombuildsstuff added this to the Blocked milestone Mar 22, 2024

tombuildsstuff added the dependent-on-another-pr label Mar 22, 2024

managedhsm: introducing dedicated Resource ID Parsers for the Data Pl…

99d4bdf

…ane Versioned and Versionless Key IDs

Botje force-pushed the storage-account-cmk branch from e993524 to 341962d Compare April 16, 2024 12:55

github-actions bot added the waiting-response label Apr 16, 2024

Botje mentioned this pull request Apr 16, 2024

managedhsm: introducing dedicated Resource ID Parsers for the Data Plane Versioned and Versionless Key IDs #25601

Merged

Botje added 3 commits April 16, 2024 15:22

azurerm_storage_account - support HSMs

1293315

azurerm_storage_account_customer_managed_key - support HSMs

9a1b37f

Extract "IsManagedHSMURI" helper

de18088

Botje force-pushed the storage-account-cmk branch from 341962d to de18088 Compare April 16, 2024 13:24

github-actions bot removed the waiting-response label Apr 16, 2024

merge main

dceb300

github-actions bot added size/L and removed size/XL labels May 2, 2024

katbyte removed this from the Blocked milestone May 2, 2024

katbyte mentioned this pull request May 2, 2024

azurerm_mssql_server_transparent_data_encryption - support for the managed_hsm_key_id property #25839

Open

14 tasks

mbfrahry added 2 commits May 2, 2024 16:28

Update storage_account_resource.go

2af83e0

err != nil -> err == nil

Update storage_account_customer_managed_key_resource.go

82b4a2e

err != nil -> err == nil

mbfrahry approved these changes May 2, 2024

View reviewed changes

katbyte approved these changes May 3, 2024

View reviewed changes

katbyte merged commit 982d373 into hashicorp:main May 3, 2024
34 checks passed

github-actions bot added this to the v3.102.0 milestone May 3, 2024

katbyte added a commit that referenced this pull request May 3, 2024

Update CHANGELOG.md #25088

72f651c

riemers mentioned this pull request May 7, 2024

can't read azure resources with manually added managedHSM encryption anymore #23321

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`azurerm_storage_account`: Support managed HSMs for CMKs #25088

`azurerm_storage_account`: Support managed HSMs for CMKs #25088

Botje commented Feb 28, 2024 •

edited by rcskosir

katbyte commented Feb 28, 2024

Botje commented Feb 29, 2024 •

edited

Botje commented Mar 1, 2024 •

edited

riemers commented Mar 6, 2024

Botje commented Mar 6, 2024

riemers commented Mar 12, 2024

Botje commented Mar 12, 2024

riemers commented Mar 12, 2024

Botje commented Mar 12, 2024

riemers commented Mar 13, 2024

tombuildsstuff commented Mar 22, 2024

Botje commented Mar 22, 2024

riemers commented Apr 24, 2024

katbyte commented May 2, 2024

mbfrahry left a comment

katbyte left a comment

azurerm_storage_account: Support managed HSMs for CMKs #25088

azurerm_storage_account: Support managed HSMs for CMKs #25088

Conversation

Botje commented Feb 28, 2024 • edited by rcskosir

katbyte commented Feb 28, 2024

Botje commented Feb 29, 2024 • edited

Botje commented Mar 1, 2024 • edited

riemers commented Mar 6, 2024

Botje commented Mar 6, 2024

riemers commented Mar 12, 2024

Botje commented Mar 12, 2024

riemers commented Mar 12, 2024

Botje commented Mar 12, 2024

riemers commented Mar 13, 2024

tombuildsstuff commented Mar 22, 2024

Botje commented Mar 22, 2024

riemers commented Apr 24, 2024

katbyte commented May 2, 2024

mbfrahry left a comment

Choose a reason for hiding this comment

katbyte left a comment

Choose a reason for hiding this comment

`azurerm_storage_account`: Support managed HSMs for CMKs #25088

`azurerm_storage_account`: Support managed HSMs for CMKs #25088

Botje commented Feb 28, 2024 •

edited by rcskosir

Botje commented Feb 29, 2024 •

edited

Botje commented Mar 1, 2024 •

edited