Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_key_vault_managed_hardware_security_module - support for activate_config #20855

Merged
merged 6 commits into from
Jun 9, 2023

Conversation

wuxu92
Copy link
Contributor

@wuxu92 wuxu92 commented Mar 9, 2023

Draft for download pending SDK issue

the managed HSM has to be activated before using it. This PR add this function by using the certificates generated from keyvault reosurce.

this function is needed for creating keys under managed HSM.

reference doc: https://learn.microsoft.com/en-us/azure/key-vault/managed-hsm/quick-create-cli#activate-your-managed-hsm

docs of az cli: https://learn.microsoft.com/en-us/cli/azure/keyvault/security-domain?view=azure-cli-latest

=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/download
        --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/resource/download (1839.39s)

(related pr #20150)

Copy link
Collaborator

@katbyte katbyte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to fix the tests up:

 testcase.go:117: Step 1/2 error: Error running apply: exit status 1
        
        Error: creating Managed H S M: (Name "kvHsm230401192111680619" / Resource Group "acctestRG-KV-230401192111680619"): keyvault.ManagedHsmsClient#CreateOrUpdate: Failure sending request: StatusCode=503 -- Original Error: Code="503" Message="Pool creation is disabled in this region"
        
          with azurerm_key_vault_managed_hardware_security_module.test,
          on terraform_plugin_test.tf line 27, in resource "azurerm_key_vault_managed_hardware_security_module" "test":
          27: resource "azurerm_key_vault_managed_hardware_security_module" "test" {
        
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/update
    testcase.go:117: Step 1/2 error: Error running apply: exit status 1
        
        Error: creating Managed H S M: (Name "kvHsm230401192433135766" / Resource Group "acctestRG-KV-230401192433135766"): keyvault.ManagedHsmsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="ManagedHsmInsufficentQuota" Message="You may only create only 5 HSM pool(s) per subscription in a region."
        
          with azurerm_key_vault_managed_hardware_security_module.test,
          on terraform_plugin_test.tf line 27, in resource "azurerm_key_vault_managed_hardware_security_module" "test":
          27: resource "azurerm_key_vault_managed_hardware_security_module" "test" {
        
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/complete
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/download
    testcase.go:117: Step 1/6 error: Error running apply: exit status 1
        
        Error: creating Managed H S M: (Name "kvHsm230401195642032955" / Resource Group "acctestRG-KV-230401195642032955"): keyvault.ManagedHsmsClient#CreateOrUpdate: Failure sending request: StatusCode=503 -- Original Error: Code="503" Message="Pool creation is disabled in this region"
        
          with azurerm_key_vault_managed_hardware_security_module.test,
          on terraform_plugin_test.tf line 27, in resource "azurerm_key_vault_managed_hardware_security_module" "test":
          27: resource "azurerm_key_vault_managed_hardware_security_module" "test" {

@wuxu92
Copy link
Contributor Author

wuxu92 commented Apr 23, 2023

acc local run pass now, I'll run in TC later.

=== RUN   TestAccKeyVaultManagedHardwareSecurityModule
--- PASS: TestAccKeyVaultManagedHardwareSecurityModule (11364.42s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/data_source
    --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/data_source (2352.79s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/data_source/basic
        --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/data_source/basic (2352.79s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource
    --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/resource (9011.63s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/update
        --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/resource/update (1775.53s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/complete
        --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/resource/complete (1876.14s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/download
        --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/resource/download (3319.75s)
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/basic
        --- PASS: TestAccKeyVaultManagedHardwareSecurityModule/resource/basic (2040.20s)
PASS

Copy link
Member

@mbfrahry mbfrahry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @wuxu92, it looks like this package got moved over to hashicorp/go-azure-sdk can you migrate this over to that sdk. I also wonder if any of the long running actions we are doing in this PR are fixed through that sdk

@wuxu92
Copy link
Contributor Author

wuxu92 commented May 23, 2023

Hi @mbfrahry I have rebased origin/main and the LRO issue exists in the new sdk too.

Copy link
Member

@mbfrahry mbfrahry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @wuxu92, thanks for getting that ported over. I've looked through again and it needs more testing and design changes that I've documented below.

@@ -130,20 +142,53 @@ func resourceKeyVaultManagedHardwareSecurityModule() *pluginsdk.Resource {
},
},

"activate_config": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this isn't ForceNew can we add a test that adds certificates after already activating the hsm and also another step that removes certificates/removes them entirely?

Copy link
Member

@tombuildsstuff tombuildsstuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments inline, but if we can fix those up then we should be able to take another look here 👍

Elem: &pluginsdk.Resource{
Schema: map[string]*schema.Schema{
"security_domain_certificate": {
Type: pluginsdk.TypeSet,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the ordering important here?

},
},

"security_domain_enc_data": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"security_domain_enc_data": {
"security_domain_encrypted_data": {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In retrospect/per the documentation below, this probably shouldn't be part of this resource, but instead a separate resource, what's the reasoning for in-lining this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an activate action rather than a resource. it has no update/delete methods so I think it should be a part of the managed HSM itself.

Comment on lines +486 to +504
originResponse := future.Response()
data, err := io.ReadAll(originResponse.Body)
if err != nil {
return "", err
}
var encData struct {
Value string `json:"value"`
}

err = json.Unmarshal(data, &encData)
if err != nil {
return "", fmt.Errorf("unmarshal EncData: %v", err)
}

err = waitHSMPendingStatus(ctx, vaultBaseURL, sdClient)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be done/exposed by go-azure-sdk - is there an issue for this?


// if isUpload is false, then check the download pending
// the generated SDK of `future.WaitForCompletionRef` not work, see:
// https://github.com/Azure/azure-rest-api-specs/issues/23035
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this shouldn't be needed with go-azure-sdk?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a data-plane API. Does go-azure-sdk support data-plane now?

Comment on lines 234 to 245
if err != nil || resp.Model == nil || resp.Model.Properties == nil || resp.Model.Properties.HsmUri == nil {
return fmt.Errorf("get nil HSMUri for %s: %+v", id, err)
} else {
encData, err := securityDomainDownload(ctx,
kvClient,
*resp.Model.Properties.HsmUri,
certs[0].(map[string]interface{}),
)
if err == nil {
d.Set("security_domain_enc_data", encData)
} else {
log.Printf("security domain download: %v", err)
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this becomes a lot simpler/less crash-prone if we pull hsmUri out into a variable:

Suggested change
if err != nil || resp.Model == nil || resp.Model.Properties == nil || resp.Model.Properties.HsmUri == nil {
return fmt.Errorf("get nil HSMUri for %s: %+v", id, err)
} else {
encData, err := securityDomainDownload(ctx,
kvClient,
*resp.Model.Properties.HsmUri,
certs[0].(map[string]interface{}),
)
if err == nil {
d.Set("security_domain_enc_data", encData)
} else {
log.Printf("security domain download: %v", err)
}
}
hsmUri := ""
if model := resp.Model; model != nil && model.Properties != nil && model.Properties.HsmUri != nil {
hsmUri = *model.Properties.HsmUri
}
if hsmUri == "" {
return fmt.Errorf("retrieving %s: `properties.HsmUri` was nil", id)
}
encryptedData, err := securityDomainDownload(ctx, kvClient, hsmUri, certs)
if err != nil {
return fmt.Errorf("downloading security domain for %s: %+v", id, err)
}
d.Set("security_domain_enc_data", encryptedData)

it's also worth calling out that we're assuming the length on certs here, which is a potential crash point - we should instead pass that into the other function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The length of certs has been checked in the if statement before.


A `activate_config` block supports the following:

* `security_domain_certificate` - (Required) A list of KeyVault certificates resource ID(minimum of three and up to a maximum of 10) to activate this Managed HSM. More information see [activate-your-managed-hsm](https://learn.microsoft.com/en-us/azure/key-vault/managed-hsm/quick-create-cli#activate-your-managed-hsm)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't use the phrase Resource ID, since it's ambiguous - this should be:

Suggested change
* `security_domain_certificate` - (Required) A list of KeyVault certificates resource ID(minimum of three and up to a maximum of 10) to activate this Managed HSM. More information see [activate-your-managed-hsm](https://learn.microsoft.com/en-us/azure/key-vault/managed-hsm/quick-create-cli#activate-your-managed-hsm)
* `security_domain_certificate` - (Required) A list of Key Vault Certificate IDs which should be used to activate this Managed HSM. More information on [activating the Managed HSM can be found in the Microsoft documentation](https://learn.microsoft.com/en-us/azure/key-vault/managed-hsm/quick-create-cli#activate-your-managed-hsm).

Also this name isn't matching what this is doing, so perhaps this field wants renaming?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you think of renaming as certificate_ids?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security_domain_certificate_ids or acrivation_key_vault_certificat_ids ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to activation_key_vault_certificate_ids

Copy link
Collaborator

@katbyte katbyte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have some test failures

Error: creating Managed H S M (Subscription: "*******"
        Resource Group Name: "acctestRG-KV-230601191510354687"
        Managed H S M Name: "kvHsm230601191510354687"): performing CreateOrUpdate: managedhsms.ManagedHsmsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="GatewayTimeout" Message="The gateway did not receive a response from 'Microsoft.KeyVault' within the specified time period."
        
          with azurerm_key_vault_managed_hardware_security_module.test,
          on terraform_plugin_test.tf line 28, in resource "azurerm_key_vault_managed_hardware_security_module" "test":
          28: resource "azurerm_key_vault_managed_hardware_security_module" "test" {
        
=== RUN   TestAccKeyVaultManagedHardwareSecurityModule/resource/basic
    testcase.go:117: Step 1/2 error: Error running apply: exit status 1
        
        Error: creating Managed H S M (Subscription: "*******"
        Resource Group Name: "acctestRG-KV-230601192459875707"
        Managed H S M Name: "kvHsm230601192459875707"): performing CreateOrUpdate: managedhsms.ManagedHsmsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="503" Message="Capacity check failed"

@wuxu92
Copy link
Contributor Author

wuxu92 commented Jun 6, 2023

@katbyte Only some regions support creating Managed HSM resources and there is a quota limitation (only 5 in my experience) for a subscription in each region. so it's easy to failed. This case can get pass easier if set env.ARM_PROVIDER_DYNAMIC_TEST as false.

image

Copy link
Member

@mbfrahry mbfrahry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for making those changes @wuxu92

@mbfrahry mbfrahry changed the title azurerm_key_vault_managed_hardware_security_module - support activate ManagedHSM in terraform azurerm_key_vault_managed_hardware_security_module - support for activate_config Jun 9, 2023
@mbfrahry mbfrahry merged commit b22ea97 into hashicorp:main Jun 9, 2023
14 checks passed
mbfrahry added a commit that referenced this pull request Jun 9, 2023
@github-actions github-actions bot added this to the v3.60.0 milestone Jun 9, 2023
mbfrahry added a commit that referenced this pull request Jun 9, 2023
mbfrahry added a commit that referenced this pull request Jun 9, 2023
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants