New MRS v2 cluster resource #1324

Lance52259 · 2021-07-30T02:06:28Z

What this PR does / why we need it:
Compared with the V1 version of the cluster, the V2 version have a more user-friendly parameter structure, and supports enterprise project ID and node expansion and contraction.

Which issue this PR fixes:
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)
fixes #1149 #1306

Special notes for your reviewer:

Release note:

1. New cluster resource support:
    a. support master, core and task nodes management
    b. support resize core groups for core and task group
2. Related document support.
3. Related acc test support (except for the failure of the ANALYSIS cluster due to the security group rule port 9022 error).

PR Checklist

Tests added/passed.
Documentation updated.
Schema updated.

Acceptance Steps Performed

basic test

make testacc
TEST='./huaweicloud/services/acceptance/mrs' TESTARGS='-run=TestAccMrsMapReduceCluster_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./huaweicloud/services/acceptance/mrs -v -run=TestAccMrsMapReduceCluster_basic -timeout 360m -parallel 4
=== RUN   TestAccMrsMapReduceCluster_basic
=== PAUSE TestAccMrsMapReduceCluster_basic
=== CONT  TestAccMrsMapReduceCluster_basic
--- PASS: TestAccMrsMapReduceCluster_basic (1352.10s)
PASS
ok      github.com/huaweicloud/terraform-provider-huaweicloud/huaweicloud/services/acceptance/mrs1352.177s

with keypair

make testacc 
TEST='./huaweicloud/services/acceptance/mrs' TESTARGS='-run=TestAccMrsMapReduceCluster_keypair'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./huaweicloud/services/acceptance/mrs -v -run=TestAccMrsMapReduceCluster_keypair -timeout 360m -parallel 4
=== RUN   TestAccMrsMapReduceCluster_keypair
=== PAUSE TestAccMrsMapReduceCluster_keypair
=== CONT  TestAccMrsMapReduceCluster_keypair
--- PASS: TestAccMrsMapReduceCluster_keypair (727.97s)
PASS
ok      github.com/huaweicloud/terraform-provider-huaweicloud/huaweicloud/services/acceptance/mrs 728.042s

- support master, core and task nodes management - support resize core groups for core and task group

ShiChangkuo · 2021-07-30T07:34:47Z

huaweicloud/services/mrs/resource_huaweicloud_mapreduce_cluster.go

+	"github.com/huaweicloud/golangsdk/openstack/mrs/v1/cluster"
+	"github.com/huaweicloud/golangsdk/openstack/mrs/v2/clusters"


as we used both v1 and v2 packages, suggest to give an alias for one of them

The v2 sdk use clusterV2 as an alias now.

huaweicloud/services/mrs/resource_huaweicloud_mapreduce_cluster.go

ShiChangkuo · 2021-07-30T07:48:11Z

huaweicloud/services/mrs/resource_huaweicloud_mapreduce_cluster.go

+				Type:     schema.TypeString,
+				Computed: true,
+			},
+			"external_ip": {


Not public IP, it's a external IP for MRS managent page and the external_ip has been removed.

huaweicloud/services/mrs/resource_huaweicloud_mapreduce_cluster.go

docs/resources/mapreduce_cluster.md

ShiChangkuo · 2021-07-30T08:21:52Z

docs/resources/mapreduce_cluster.md

+* `subnet_id` - (Required, String, ForceNew) Specifies the network ID of a subnet.
+  Changing this will create a new MapReduce cluster resource.
+
+  -> **NOTE:** The duplication of subnet CIDR will cause the MRS cluster creation to fail.


can we create a subent with duplication CIDR?

the description has been removed.

ShiChangkuo · 2021-07-30T08:24:18Z

docs/resources/mapreduce_cluster.md

+* `external_ip` - The external IP address.
+* `private_ip` - The preferred private IP address.
+* `internal_ip` - The internal IP address.
+* `external_alternate_ip` - IP address of a Master node.


what's the differents between xx_ip ?

The private_ip is the virtual ip of the HA master node of the master node group.
The external_ip, external_alternate_ip and internal_ip are not used, has been removed.

docs/resources/mapreduce_cluster.md

Lance52259 mentioned this pull request Jul 30, 2021

Error creating MRS #1290

Closed

feat: new mrs v2 cluster supported

6538341

- support master, core and task nodes management - support resize core groups for core and task group

ShiChangkuo suggested changes Jul 30, 2021

View reviewed changes

Lance52259 mentioned this pull request Jul 30, 2021

New MRS v2 job resource #1327

Merged

3 tasks

update provider according to suggestions

8ba80c5

ShiChangkuo approved these changes Jul 31, 2021

View reviewed changes

ShiChangkuo merged commit 3071043 into huaweicloud:master Jul 31, 2021

Lance52259 deleted the br_mrs_cluster_active branch August 31, 2021 09:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New MRS v2 cluster resource #1324

New MRS v2 cluster resource #1324

Lance52259 commented Jul 30, 2021 •

edited

Loading

ShiChangkuo Jul 30, 2021

Lance52259 Jul 31, 2021

ShiChangkuo Jul 30, 2021

Lance52259 Jul 31, 2021 •

edited

Loading

ShiChangkuo Jul 30, 2021

Lance52259 Jul 31, 2021 •

edited

Loading

ShiChangkuo Jul 30, 2021

Lance52259 Jul 31, 2021 •

edited

Loading

		"github.com/huaweicloud/golangsdk/openstack/mrs/v1/cluster"
		"github.com/huaweicloud/golangsdk/openstack/mrs/v2/clusters"

New MRS v2 cluster resource #1324

New MRS v2 cluster resource #1324

Conversation

Lance52259 commented Jul 30, 2021 • edited Loading

PR Checklist

Acceptance Steps Performed

ShiChangkuo Jul 30, 2021

Choose a reason for hiding this comment

Lance52259 Jul 31, 2021

Choose a reason for hiding this comment

ShiChangkuo Jul 30, 2021

Choose a reason for hiding this comment

Lance52259 Jul 31, 2021 • edited Loading

Choose a reason for hiding this comment

ShiChangkuo Jul 30, 2021

Choose a reason for hiding this comment

Lance52259 Jul 31, 2021 • edited Loading

Choose a reason for hiding this comment

ShiChangkuo Jul 30, 2021

Choose a reason for hiding this comment

Lance52259 Jul 31, 2021 • edited Loading

Choose a reason for hiding this comment

Lance52259 commented Jul 30, 2021 •

edited

Loading

Lance52259 Jul 31, 2021 •

edited

Loading

Lance52259 Jul 31, 2021 •

edited

Loading

Lance52259 Jul 31, 2021 •

edited

Loading