F/aws emr cluster #17706

nikhil-goenka · 2021-02-19T14:50:56Z

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates OR Closes #17062

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAWSEMRCluster_s3Logging'
=== RUN   TestAccAWSEMRCluster_s3Logging
=== PAUSE TestAccAWSEMRCluster_s3Logging
=== CONT  TestAccAWSEMRCluster_s3Logging
--- PASS: TestAccAWSEMRCluster_s3Logging (649.24s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	655.676s
$ make testacc TESTARGS='-run=TestAccAWSEMRCluster_s3LogEncryption'
=== RUN   TestAccAWSEMRCluster_s3LogEncryption
=== PAUSE TestAccAWSEMRCluster_s3LogEncryption
=== CONT  TestAccAWSEMRCluster_s3LogEncryption
--- PASS: TestAccAWSEMRCluster_s3LogEncryption (792.65s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	797.210s
...

pradeepbhadani · 2021-02-19T15:10:23Z

website/docs/r/emr_cluster.html.markdown

@@ -325,6 +325,7 @@ The following arguments are supported:
 * `security_configuration` - (Optional) The security configuration name to attach to the EMR cluster. Only valid for EMR clusters with `release_label` 4.8.0 or greater
 * `core_instance_group` - (Optional) Configuration block to use an [Instance Group](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-group-configuration.html#emr-plan-instance-groups) for the [core node type](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-master-core-task-nodes.html#emr-plan-core).
 * `core_instance_fleet` - (Optional) Configuration block to use an [Instance Fleet](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-instance-fleet.html) for the core node type. Cannot be specified if any `core_instance_group` configuration blocks are set. Detailed below.
+* `log_encryption_kms_key_id` - (Optional) The AWS KMS master key ID used for encrypting log files. This attribute is only available with EMR version 5.30.0 and later, excluding EMR 6.0.0.  


As per https://docs.aws.amazon.com/emr/latest/APIReference/API_JobFlowDetail.html,
this is Customer Master Key (CMK). Could you please replace master key with customer master key (CMK) to align with AWS documentation?

Thanks, Updated

Thanks. Looks good to me.

bflad · 2021-02-19T20:04:30Z

aws/resource_aws_emr_cluster_test.go

@@ -2970,16 +2995,17 @@ resource "aws_emr_cluster" "tf-test-cluster" {
    instance_type  = "c4.large"
  }

-  log_uri = "s3://${aws_s3_bucket.test.bucket}/"


Hi @nikhil-goenka 👋 Thank you for submitting this enhancement. Can you please create a new acceptance test and test configuration to verify this change instead of modifying the existing one? This helps us ensure there are no unexpected regressions. Please see also: https://github.com/hashicorp/terraform-provider-aws/blob/main/docs/contributing/contribution-checklists.md#enhancementbugfix-to-a-resource

Got the logic, updated

karthik-hbo · 2021-07-10T20:24:29Z

Hi Team,
When this will be merged and available for use?

lokm01 · 2021-08-10T10:21:18Z

Hi all, any updates on this please?

lokm01 · 2021-08-26T07:27:33Z

@bflad @pradeepbhadani any chance to review this again please? :)

Zejnilovic · 2021-08-26T09:14:58Z

Hello guys. Is there any new information about this? Any update is appreciated.

pradeepbhadani · 2021-08-27T15:58:11Z

@bflad @pradeepbhadani any chance to review this again please? :)

I am not the approver on this project. but the change looks good to me. Reaching out to HashiCorp Community on getting this reviewed.

YakDriver

Looks good! 👍

Output of acceptance tests (us-west-2):

--- PASS: TestAccAWSEMRCluster_basic (394.16s)
--- PASS: TestAccAWSEMRCluster_Step_Basic (495.96s)
--- PASS: TestAccAWSEMRCluster_InstanceFleet_master_only (522.43s)
--- PASS: TestAccAWSEMRCluster_security_config (540.73s)
--- PASS: TestAccAWSEMRCluster_step_concurrency_level (560.61s)
--- PASS: TestAccAWSEMRCluster_ebs_config (575.45s)
--- PASS: TestAccAWSEMRCluster_custom_ami_id (579.43s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy (677.97s)
--- PASS: TestAccAWSEMRCluster_s3LogEncryption (701.55s)
--- PASS: TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc (720.66s)
--- PASS: TestAccAWSEMRCluster_Ec2Attributes_DefaultManagedSecurityGroups (864.27s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType (888.21s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Name (914.91s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Name (965.39s)
--- PASS: TestAccAWSEMRCluster_keepJob (422.17s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice (983.19s)
--- PASS: TestAccAWSEMRCluster_root_volume_size (1028.63s)
--- PASS: TestAccAWSEMRCluster_terminationProtected (560.45s)
--- PASS: TestAccAWSEMRCluster_s3Logging (541.94s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice (1065.99s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceCount (1095.81s)
--- PASS: TestAccAWSEMRCluster_disappears (521.71s)
--- PASS: TestAccAWSEMRCluster_configurationsJson (520.61s)
--- PASS: TestAccAWSEMRCluster_additionalInfo (400.55s)
--- PASS: TestAccAWSEMRCluster_Step_Multiple (476.43s)
--- PASS: TestAccAWSEMRCluster_InstanceFleet_basic (1260.41s)
--- PASS: TestAccAWSEMRCluster_tags (873.52s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount (1278.96s)
--- PASS: TestAccAWSEMRCluster_visibleToAllUsers (805.84s)
--- PASS: TestAccAWSEMRCluster_Step_ConfigMode (859.46s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType (906.93s)
--- PASS: TestAccAWSEMRCluster_bootstrap_ordering (1277.84s)

Output of acceptance tests (GovCloud):

(Failures unrelated to these changes and stem from differences on GovCloud downloading bootstrap action.)

--- FAIL: TestAccAWSEMRCluster_s3Logging (262.62s)
--- FAIL: TestAccAWSEMRCluster_InstanceFleet_master_only (310.53s)
--- FAIL: TestAccAWSEMRCluster_InstanceFleet_basic (322.48s)
--- PASS: TestAccAWSEMRCluster_Kerberos_ClusterDedicatedKdc (399.96s)
--- PASS: TestAccAWSEMRCluster_ebs_config (474.33s)
--- PASS: TestAccAWSEMRCluster_custom_ami_id (480.35s)
--- PASS: TestAccAWSEMRCluster_Step_Multiple (488.69s)
--- PASS: TestAccAWSEMRCluster_step_concurrency_level (497.75s)
--- FAIL: TestAccAWSEMRCluster_bootstrap_ordering (275.12s)
--- PASS: TestAccAWSEMRCluster_Step_Basic (548.95s)
--- PASS: TestAccAWSEMRCluster_security_config (563.88s)
--- PASS: TestAccAWSEMRCluster_basic (580.93s)
--- PASS: TestAccAWSEMRCluster_keepJob (397.85s)
--- PASS: TestAccAWSEMRCluster_visibleToAllUsers (823.01s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_Name (831.01s)
--- PASS: TestAccAWSEMRCluster_root_volume_size (857.41s)
--- PASS: TestAccAWSEMRCluster_terminationProtected (541.65s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_Name (873.37s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceType (883.61s)
--- PASS: TestAccAWSEMRCluster_tags (923.43s)
--- PASS: TestAccAWSEMRCluster_Step_ConfigMode (933.06s)
--- PASS: TestAccAWSEMRCluster_additionalInfo (407.19s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceCount (505.34s)
--- PASS: TestAccAWSEMRCluster_configurationsJson (446.65s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_AutoscalingPolicy (564.35s)
--- PASS: TestAccAWSEMRCluster_disappears (500.07s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_InstanceCount (1076.35s)
--- PASS: TestAccAWSEMRCluster_Ec2Attributes_DefaultManagedSecurityGroups (762.44s)
--- PASS: TestAccAWSEMRCluster_s3LogEncryption (1166.72s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_InstanceType (761.26s)
--- PASS: TestAccAWSEMRCluster_MasterInstanceGroup_BidPrice (822.30s)
--- PASS: TestAccAWSEMRCluster_CoreInstanceGroup_BidPrice (950.06s)

github-actions · 2021-10-08T00:59:22Z

This functionality has been released in v3.62.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions · 2022-06-17T02:35:19Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

nikhil-goenka requested a review from a team as a code owner February 19, 2021 14:50

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Feb 19, 2021

pradeepbhadani reviewed Feb 19, 2021

View reviewed changes

bflad reviewed Feb 19, 2021

View reviewed changes

ghost added size/M Managed by automation to categorize the size of a PR. and removed size/S Managed by automation to categorize the size of a PR. labels Feb 20, 2021

nikhil-goenka requested a review from bflad May 1, 2021 04:14

breathingdust added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Sep 8, 2021

YakDriver self-assigned this Oct 5, 2021

nikhil-goenka added 7 commits October 5, 2021 14:38

LogEncryptionKmsKeyId for EMR Clusters

058ad4e

LogEncryptionKmsKeyId for EMR Clusters

bcde90d

LogEncryptionKmsKeyId for EMR Clusters

fb76a88

LogEncryptionKmsKeyId for EMR Clusters

f01295a

LogEncryptionKmsKeyId for EMR Clusters

31daddd

LogEncryptionKmsKeyId for EMR Clusters

85a29b5

LogEncryptionKmsKeyId for EMR Clusters

547aa55

YakDriver force-pushed the f/aws_emr_cluster branch from 21b307c to ea9f0cf Compare October 5, 2021 18:43

nikhil-goenka and others added 2 commits October 5, 2021 14:47

LogEncryptionKmsKeyId for EMR Clusters

1b45935

tests/emr_cluster: Clean-up, standardize tests

b8e0055

YakDriver force-pushed the f/aws_emr_cluster branch from ea9f0cf to b8e0055 Compare October 5, 2021 21:11

github-actions bot added size/XL Managed by automation to categorize the size of a PR. and removed size/M Managed by automation to categorize the size of a PR. labels Oct 5, 2021

docs/emr_cluster: Clean up docs

45e76ad

YakDriver approved these changes Oct 5, 2021

View reviewed changes

YakDriver added this to the v3.62.0 milestone Oct 5, 2021

YakDriver merged commit a3d854a into hashicorp:main Oct 5, 2021

github-actions bot locked as resolved and limited conversation to collaborators Jun 17, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

F/aws emr cluster #17706

F/aws emr cluster #17706

nikhil-goenka commented Feb 19, 2021 •

edited

pradeepbhadani Feb 19, 2021

nikhil-goenka Feb 19, 2021

pradeepbhadani Aug 27, 2021

bflad Feb 19, 2021

nikhil-goenka Feb 20, 2021

karthik-hbo commented Jul 10, 2021

lokm01 commented Aug 10, 2021

lokm01 commented Aug 26, 2021

Zejnilovic commented Aug 26, 2021

pradeepbhadani commented Aug 27, 2021

YakDriver left a comment •

edited

github-actions bot commented Oct 8, 2021

github-actions bot commented Jun 17, 2022

F/aws emr cluster #17706

F/aws emr cluster #17706

Conversation

nikhil-goenka commented Feb 19, 2021 • edited

Community Note

pradeepbhadani Feb 19, 2021

Choose a reason for hiding this comment

nikhil-goenka Feb 19, 2021

Choose a reason for hiding this comment

pradeepbhadani Aug 27, 2021

Choose a reason for hiding this comment

bflad Feb 19, 2021

Choose a reason for hiding this comment

nikhil-goenka Feb 20, 2021

Choose a reason for hiding this comment

karthik-hbo commented Jul 10, 2021

lokm01 commented Aug 10, 2021

lokm01 commented Aug 26, 2021

Zejnilovic commented Aug 26, 2021

pradeepbhadani commented Aug 27, 2021

YakDriver left a comment • edited

Choose a reason for hiding this comment

github-actions bot commented Oct 8, 2021

github-actions bot commented Jun 17, 2022

nikhil-goenka commented Feb 19, 2021 •

edited

YakDriver left a comment •

edited