Test review and cleanup

[Didainius]

This PR aims to cover any testing issues that were found in pre-release testing.

A list of discovered and fixed things:

• Switches default VDC for binary tests (and future upgrade tests) to NSX-T VDC instead of NSX-V (aligns with acceptance test work done in Convert tests to use NSX-T based VDC #886)

• Patches make test-binary (in file scripts/runtest.sh) to avoid running outdated templates in test-templates directory by default

• Adds a test TestAccVcdNsxVdcGroupResources (commit dcf8ac1) which hits lock errors when Edge Gateway is part of VDC Group. This is fixed in commit 73d008b by using a conditional locking mechanism:

  • lock on Edge Gateway when it is in VDC
  • lock on VDC Group when Edge Gateway is a member of VDC Group.
    Note. This locking mechanism copy/pasted from NSX-T BGP and Route Advertisement resources.

Sample error:

error creating NSX-T Firewall Rules: error setting NSX-T Firewall: error in HTTP PUT request: BUSY_ENTITY - [ 00c269de-e89d-4478-91ac-03d0408effd1 ] The entity Ref: com.vmware.vcloud.entity.gateway:64d910fd-69c4-4d0f-95ca-3c8635a194ad is busy completing an
 operation CROSS_VDC_NETWORK_CREATE. CROSS_VDC_NETWORK_CREATE(com.vmware.vcloud.entity.task:c0b96fb2-8e19-4819-bc76-66d113deb304)

• vcd_inserted_media and vcd_vm_internal_disk lock alignment. PR Update insert media locking to be concise with internal disk locking #884 updated vcd_inserted_media to use locks on parent VM to solve Destroy-time error when vcd_vm_internal_disk and vcd_inserted_media used together #868. However other problems occurred because of that (sample error below) as other resource (vapp network) used lock at vApp level and we got conflict. Apparently mounting media requires lock at vApp level. So to avoid locks at different levels (vApp vs VM), resource vcd_vm_internal_disk is now also switched to lock at vApp level.

Sample error:

vcd_vapp_vm.tf_vm_second: Still destroying... [id=urn:vcloud:vm:6b7f65dd-81a6-4611-b558-34e5c7075590, 50s elapsed]
vcd_vapp_vm.tf_vm_second: Destruction complete after 56s
╷
│ Warning: Quoted references are deprecated
│
│   on config.tf line 64, in resource "vcd_vapp" "tf_vapp":
│   64:   depends_on = ["vcd_network_routed.tf_network"]
│
│ In this context, references are expected literally rather than in quotes. Terraform 0.11 and earlier required quotes, but quoted
│ references are now deprecated and will be removed in a future version of Terraform. Remove the quotes surrounding this reference to
│ silence this warning.
╵
╷
│ Error: error: &errors.errorString{s:"error insert or eject media: API Error: 400: [ 140c3816-fa7d-48ad-a51e-e3adf44cccb5 ] The entity Ref: com.vmware.vcloud.entity.vm:03a0d22d-74cd-493a-8646-e8265b1f2b21 is busy completing an operation VDC_UPDATE_VAPP_NETWORK_SECTION. VDC_UPDATE_VAPP_NETWORK_SECTION(com.vmware.vcloud.entity.task:be2728f5-c411-4483-9ced-af109448f8cd)"}
│

• Failing acceptance test fixes:
  • TestAccVcdNsxtAlbVdcGroupIntegration and a few other ALB tests sometimes hit errors as below on VCD 10.2. All tasks are being tracked, but it sounds like VCD needs another second after the infrastructure part of configuration before enabling it on particular Edge Gateway (T1). This patch gives additional 5 second sleep between infrastructure and tenant configuration parts. Only for this test for validation. I have ran it 5 times on 3 envs and it did not occur.

 resource_vcd_nsxt_alb_common_test.go:170: Step 1/7 error: Error running apply: exit status 1
        
        Error: error setting NSX-T ALB General Settings: error waiting completion of task (https://HOSTNAME/api/task/cf2d55cb-ab6a-4ff8-acda-ca6e975f5d39): task did not complete successfully:  [500:INTERNAL_SERVER_ERROR] - [ 332c81cf-1803-419c-8688-a78b6fa284b3 ] Load Balancer Cloud is not in ready state when enabling load balancer for Edge Gateway TestAccVcdNsxtAlbVdcGroupIntegration with Tier-1 95cc7d45-e65f-4e16-ba33-a48cb189a6f4. Cloud has state CLOUD_STATE_IN_PROGRESS and reason: .
         - VrfContext is not ready yet for Edge Gateway TestAccVcdNsxtAlbVdcGroupIntegration with Tier-1 95cc7d45-e65f-4e16-ba33-a48cb189a6f4.
        
          with vcd_nsxt_alb_settings.test,
          on terraform_plugin_test.tf line 76, in resource "vcd_nsxt_alb_settings" "test":
          76: resource "vcd_nsxt_alb_settings" "test" {
        
    testing_new.go:84: Error running post-test destroy, there may be dangling resources: 1 error occurred:
        	* Check 5/5 error: not found resource: vcd_nsxt_alb_virtual_service.test
        

• These acceptance tests need to be skipped in org user mode:
  • FAIL: TestAccDataSourceNotFound/vcd_nsxt_edgegateway_bgp_ip_prefix_list (8.52s)
  • FAIL: TestAccDataSourceNotFound/vcd_nsxt_edgegateway_bgp_neighbor (8.57s)
  • TestAccVcdNsxtRouteAdvertisement
  • TestAccVcdNsxtRouteAdvertisementVdcGroup

[lvirbalas]

Thank you for all these important adjustments.

[adambarreiro]

Great effort, thanks a lot!