New Resource: `azurerm_kusto_cluster_customer_managed_key` #7520

jrauschenbusch · 2020-06-29T11:21:12Z

Provides Kusto Cluster support for a customer managed encryption key

mbfrahry

Hey @jrauschenbusch, taking a look through this resource, I have to say it seems better suited to just be used as an attribute on the existing cluster resource. I probably missing something so do you mind talking through a little of your thoughts around why this is split out into its own resource?

jrauschenbusch · 2020-07-01T06:39:44Z

Hey @mbfrahry,

there was a discussion with @tombuildsstuff that first the cluster needs to be created with an identity type SystemAssigned. After the cluster is created the managed identity needs to get access to the key vault. Hence this feature needs it's own resource. For further details see #7420.

jrauschenbusch · 2020-07-09T05:09:47Z

@mbfrahry @tombuildsstuff If we could get this merged today, release 2.18.0 would contain all major Kusto Cluster features currently provided by the SDK. Especially for enterprise users custom encryption keys for encryption at rest are a must.

mbfrahry

Hey @jrauschenbusch, I'm seeing errors when running the tests

TestAccAzureRMKustoClusterCustomerManagedKey_basic: testing.go:684: Step 0 error: errors during apply:
              
              Error: Error retrieving Kusto Cluster "acctestkcun8z6" (Resource Group "acctestRG-200709182537441411"): `properties` was nil

I'm investigating more on my side but any help from your end would be appreciated too

jrauschenbusch · 2020-07-09T19:20:47Z

Hey @mbfrahry This commit should do the trick. I'll test it within my own subscription and give you feedback if tests are passing.

jrauschenbusch · 2020-07-09T20:09:59Z

@mbfrahry I'm currently facing the following problems when running the acc tests:

--- FAIL: TestAccAzureRMKustoClusterCustomerManagedKey_requiresImport (803.10s)
    testing.go:684: Step 0 error: errors during apply:
        
        Error: Provider produced inconsistent result after apply
        
        When applying changes to azurerm_kusto_cluster_customer_managed_key.test,
        provider "azurerm" produced an unexpected new value for was present, but now
        absent.
        
        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.
        
    testing.go:745: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.
        
        Error: config is invalid: 4 problems:
        
        - Reference to undeclared resource: A managed resource "azurerm_storage_account_customer_managed_key" "test" has not been declared in the root module.
        - Reference to undeclared resource: A managed resource "azurerm_storage_account_customer_managed_key" "test" has not been declared in the root module.
        - Reference to undeclared resource: A managed resource "azurerm_storage_account_customer_managed_key" "test" has not been declared in the root module.
        - Reference to undeclared resource: A managed resource "azurerm_storage_account_customer_managed_key" "test" has not been declared in the root module.
...

--- FAIL: TestAccAzureRMKustoClusterCustomerManagedKey_basic (815.34s)
    testing.go:684: Step 0 error: errors during apply:
        
        Error: Provider produced inconsistent result after apply
        
        When applying changes to azurerm_kusto_cluster_customer_managed_key.test,
        provider "azurerm" produced an unexpected new value for was present, but now
        absent.
        
        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.
        
    testing.go:745: Error destroying resource! WARNING: Dangling resources
        may exist. The full state and error is shown below.
        
        Error: errors during apply: Error deleting Kusto Cluster "acctestXXXXX" (Resource Group "acctestRG-XXXXX"): kusto.ClustersClient#Delete: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> Code="RequestConflict" Message="Cannot modify resource with id '/subscriptions/XXXXXXXXXXXXXXX/resourceGroups/acctestRG-XXXXX/providers/Microsoft.Kusto/Clusters/acctestXXXXX' because the resource entity provisioning state is not terminal. Please wait for the provisioning state to become terminal and then retry the request."

--- FAIL: TestAccAzureRMKustoClusterCustomerManagedKey_updateKey (1464.67s)
    testing.go:684: Step 0 error: errors during apply:
        
        Error: Provider produced inconsistent result after apply
        
        When applying changes to azurerm_kusto_cluster_customer_managed_key.test,
        provider "azurerm" produced an unexpected new value for was present, but now
        absent.
        
        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.

Maybe you have a quick idea of where they come from?

mbfrahry · 2020-07-09T20:15:12Z

So it looks like the import test is failing because we're calling azurerm_storage_account_customer_managed_key instead of azurerm_kusto_cluster_customer_managed_key

mbfrahry · 2020-07-09T20:16:46Z

The inconsistent results failure normally happens when what is in the config differs from what is being returned by the api. That one will require more digging into. Your best bet might be to see what is being returned by the api and compare it to what's in the config

mbfrahry · 2020-07-09T20:20:11Z

I'm building it locally now to see if I can help get to the bottom of it

mbfrahry · 2020-07-09T22:32:40Z

Hmmm, so I have a bit more information. We're definitely sending the key vault properties to the api but we're not getting them back and then looking in the portal they aren't there either.

This is what we're sending with values removed:

"keyVaultProperties": {
	"keyName": "test",
	"keyVersion": "asdfqwer",
	"keyVaultUri": "hasdqweqwe"
},

And this is what we get back:

"keyVaultProperties": null,

mbfrahry · 2020-07-09T22:44:57Z

Another lead! So it looks like clusterClient.Update(ctx, clusterID.ResourceGroup, clusterID.Name, props) returns a future. I added a future wait for completion but am now seeing

Error: Code="Failed" Message="Internal Server Error"

Doing this in the portal returns the same error message

jrauschenbusch · 2020-07-10T06:32:18Z

Thanks for giving the hints. I'm going to fix it.

jrauschenbusch · 2020-07-10T07:44:32Z

Hey @mbfrahry

Regarding the Error: Code="Failed" Message="Internal Server Error":

I've now tested a lot with the resource itself and with the Azure portal, and it seems that there is currently an issue on the Azure API. I'll file a bug ticket for it.

jrauschenbusch · 2020-07-17T13:17:08Z

Hey @mbfrahry Finally the Kusto product team has fixed the bug related to the Internal Server Error message. I've finally fixed some minor bugs within the resource implementation and acc tests and now it seems to look good.

jrauschenbusch · 2020-07-17T16:18:15Z

Acceptance tests succeed now 👍

make acctests SERVICE='kusto' TESTARGS='-run=TestAccAzureRMKustoClusterCustomerManagedKey' TESTTIMEOUT='60m'
==> Checking that code complies with gofmt requirements...
==> Checking that Custom Timeouts are used...
==> Checking that acceptance test packages are used...
TF_ACC=1 go test -v ./azurerm/internal/services/kusto/tests/ -run=TestAccAzureRMKustoClusterCustomerManagedKey -timeout 60m -ldflags="-X=github.com/terraform-providers/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccAzureRMKustoClusterCustomerManagedKey_basic
=== PAUSE TestAccAzureRMKustoClusterCustomerManagedKey_basic
=== RUN   TestAccAzureRMKustoClusterCustomerManagedKey_requiresImport
=== PAUSE TestAccAzureRMKustoClusterCustomerManagedKey_requiresImport
=== RUN   TestAccAzureRMKustoClusterCustomerManagedKey_updateKey
=== PAUSE TestAccAzureRMKustoClusterCustomerManagedKey_updateKey
=== CONT  TestAccAzureRMKustoClusterCustomerManagedKey_basic
=== CONT  TestAccAzureRMKustoClusterCustomerManagedKey_updateKey
=== CONT  TestAccAzureRMKustoClusterCustomerManagedKey_requiresImport
--- PASS: TestAccAzureRMKustoClusterCustomerManagedKey_basic (1584.25s)
--- PASS: TestAccAzureRMKustoClusterCustomerManagedKey_updateKey (1651.15s)
--- PASS: TestAccAzureRMKustoClusterCustomerManagedKey_requiresImport (1692.00s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/kusto/tests 1693.087s

mbfrahry

LGTM! Thanks @jrauschenbusch!

ghost · 2020-08-17T14:50:19Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

ghost added size/XL documentation labels Jun 29, 2020

jrauschenbusch mentioned this pull request Jun 29, 2020

Kusto Cluster Key Vault support #7420

Closed

jrauschenbusch changed the title ~~New Resource: kusto_cluster_customer_managed_key~~ New Resource: azurerm_kusto_cluster_customer_managed_key Jun 29, 2020

New Resource: kusto_cluster_customer_managed_key

14c7cc7

jrauschenbusch force-pushed the azurerm_kusto_cluster_customer_managed_key branch from f3c9f84 to 14c7cc7 Compare June 30, 2020 05:24

mbfrahry added service/kusto new-resource labels Jun 30, 2020

mbfrahry reviewed Jun 30, 2020

View reviewed changes

mbfrahry added 2 commits July 1, 2020 08:33

Update kusto_cluster_customer_managed_key_test.go

ea8b480

Update kusto_cluster_customer_managed_key_test.go

075050d

mbfrahry and others added 2 commits July 9, 2020 10:09

Merge branch 'master' into azurerm_kusto_cluster_customer_managed_key

2a8a223

fix kusto resources registration

c5939e2

mbfrahry requested changes Jul 9, 2020

View reviewed changes

fix nil reference check

a47479d

fix copy/paste error

7c26de3

fix wait for future

4c0138b

jrauschenbusch added 2 commits July 17, 2020 14:39

fix: least permissions

4ac6f29

fix permissions

6798aef

jrauschenbusch added 2 commits July 17, 2020 15:09

fix import error

11072b4

fix acc tests

18e06d4

jrauschenbusch added 4 commits July 17, 2020 15:57

fix acc tests

81af7c1

fix acc test

aec4d41

refactor function name

6ec6d3c

refactor acc tests

bf62290

mbfrahry approved these changes Jul 17, 2020

View reviewed changes

mbfrahry merged commit a87636f into hashicorp:master Jul 17, 2020

mbfrahry added a commit that referenced this pull request Jul 17, 2020

Update changelog for #7520

89ce51e

jrauschenbusch deleted the azurerm_kusto_cluster_customer_managed_key branch July 18, 2020 09:46

hashicorp locked and limited conversation to collaborators Aug 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Resource: `azurerm_kusto_cluster_customer_managed_key` #7520

New Resource: `azurerm_kusto_cluster_customer_managed_key` #7520

jrauschenbusch commented Jun 29, 2020 •

edited

mbfrahry left a comment

jrauschenbusch commented Jul 1, 2020

jrauschenbusch commented Jul 9, 2020

mbfrahry left a comment

jrauschenbusch commented Jul 9, 2020

jrauschenbusch commented Jul 9, 2020

mbfrahry commented Jul 9, 2020

mbfrahry commented Jul 9, 2020 •

edited

mbfrahry commented Jul 9, 2020

mbfrahry commented Jul 9, 2020

mbfrahry commented Jul 9, 2020

jrauschenbusch commented Jul 10, 2020

jrauschenbusch commented Jul 10, 2020

jrauschenbusch commented Jul 17, 2020

jrauschenbusch commented Jul 17, 2020

mbfrahry left a comment

ghost commented Aug 17, 2020

New Resource: azurerm_kusto_cluster_customer_managed_key #7520

New Resource: azurerm_kusto_cluster_customer_managed_key #7520

Conversation

jrauschenbusch commented Jun 29, 2020 • edited

mbfrahry left a comment

Choose a reason for hiding this comment

jrauschenbusch commented Jul 1, 2020

jrauschenbusch commented Jul 9, 2020

mbfrahry left a comment

Choose a reason for hiding this comment

jrauschenbusch commented Jul 9, 2020

jrauschenbusch commented Jul 9, 2020

mbfrahry commented Jul 9, 2020

mbfrahry commented Jul 9, 2020 • edited

mbfrahry commented Jul 9, 2020

mbfrahry commented Jul 9, 2020

mbfrahry commented Jul 9, 2020

jrauschenbusch commented Jul 10, 2020

jrauschenbusch commented Jul 10, 2020

jrauschenbusch commented Jul 17, 2020

jrauschenbusch commented Jul 17, 2020

mbfrahry left a comment

Choose a reason for hiding this comment

ghost commented Aug 17, 2020

New Resource: `azurerm_kusto_cluster_customer_managed_key` #7520

New Resource: `azurerm_kusto_cluster_customer_managed_key` #7520

jrauschenbusch commented Jun 29, 2020 •

edited

mbfrahry commented Jul 9, 2020 •

edited