Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add supported services #308

Merged
merged 21 commits into from
Aug 3, 2023
Merged

feat: add supported services #308

merged 21 commits into from
Aug 3, 2023

Conversation

rajatagarwal-ibm
Copy link
Member

@rajatagarwal-ibm rajatagarwal-ibm commented May 17, 2023

Description

fixes #307

  • Add supported services for the VPE

    1. Account Management
    2. Billing
    3. Code Engine
    4. Direct Link
    5. Databases for Cassandra
    6. Databases for Elasticsearch
    7. Databases for Mongo
    8. Databases for Redis
    9. DNS Service
    10. Enterprise
    11. Global Catalog
    12. Global Search Tagging
    13. Hyper Protect DBaaS for MongoDB
    14. Hyper Protect DBaaS for PostgreSQL
    15. Identity and Access Management
    16. IAM Identity
    17. Secrets Manager
    18. Resource Controller
    19. Transit
    20. User Management
    21. VPC API
  • Could not find the following services in the UI and cannot find the VPE endpoints for them:

    1. App Configuration
    2. Catalog Management
    3. Enterprise Billing Units (Billing is added - not sure if they are the same)
    4. Enterprise Usage Report
    5. Event Notifications
    6. Event Streams for IBM Cloud
    7. Global Catalog
    8. IAM Access Groups
    9. IAM Policy Management
    10. Resource Manager
    11. Schematics
    12. Usage Metering
    13. Usage Reports
    14. Watson Assistant
    15. Watson Discovery
    16. Watson Speech to Text
    17. Watson Text to Speech
  • Removed validation from the example as that's already covered in the main module.

#307

Types of changes in this PR

Changes that affect the core Terraform module or submodules

  • Bug fix
  • New feature
  • Dependency update

Changes that don't affect the core Terraform module or submodules

  • Examples or tests (addition or updates of examples or tests)
  • Documentation update
  • CI-related update (pipeline, etc.)
  • Other

Release required?

Identify the type of release. For information about the changes in a semantic versioning release, see Release versioning.

  • No release
  • Patch release (x.x.X)
  • Minor release (x.X.x)
  • Major release (X.x.x)
Release notes content

If a release is required, replace this text with information that users need to know about the release. Write the release notes to help users understand the changes, and include information about how to update from the previous version.

Your notes helps the merger write the commit message for the PR that is published in the release notes for the module.


Checklist for reviewers

  • If relevant, a test for the change is included or updated with this PR.
  • If relevant, documentation for the change is included or updated with this PR.

Merge actions for mergers

  • Use a relevant conventional commit message that is based on the PR contents and any release notes provided by the PR author. The commit message determines whether a new version of the module is needed, and if so, which semver increment to use (major, minor, or patch).
  • Merge by using "Squash and merge".

Copy link
Member

@ocofaigh ocofaigh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update the tests to ensure we tests all of them?

@rajatagarwal-ibm
Copy link
Member Author

@ocofaigh I tried locally testing all of them but they are quite unstable when you try to create/destroy each one of them together - most of them are created/destroyed in seconds. I think that might be because each one of them is associated with the same VPC and resource group. Even with only 2 services testing they are hit and miss, especially observed while destroying.

If you say I can create a separate task to investigate further if there is a possibility to do that maybe by dividing a bunch of services across multiple VPCs or adding a timeout.

@rajatagarwal-ibm
Copy link
Member Author

rajatagarwal-ibm commented May 19, 2023

Further investigation on the error in this run https://github.com/terraform-ibm-modules/terraform-ibm-vpe-module/actions/runs/5012951676:

Will be Created

module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]
will be created
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]
will be created
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]
will be created
 
module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]
will be created
 
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]
will be created
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]
will be created
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]
will be created

Creation Phase

module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]: Creating…
module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]: Still creating... [10s
elapsed]
module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]: Creation complete after
16s [id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a]
 
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Creating…
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Still
creating... [10s elapsed]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Creation
complete after 11s
[id=0717-d10b9267-e847-462c-9ea3-3ea13ec50790/0717-b538d156-35b1-43b3-9f0c-c102cb3dff6c]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Creating…
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Creating…
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Still
creating... [10s elapsed]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Creation
complete after 10s
[id=0727-3479487d-23ad-456c-b274-df0029e4ca6b/0727-838d6e2f-5379-493e-b341-d8a9ec75df64]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Still
creating... [10s elapsed]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Creation
complete after 11s
[id=0737-b736bae7-80fa-462e-94f1-a99405c89b24/0737-b374e7c5-29f8-49a8-9da9-8c75a0fbdc98]
 
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Creating…
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Creation
complete after 5s
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0737-b374e7c5-29f8-49a8-9da9-8c75a0fbdc98]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Creating…
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Still
creating... [10s elapsed]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Creation
complete after 10s
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0717-b538d156-35b1-43b3-9f0c-c102cb3dff6c]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Creating…
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Still
creating... [10s elapsed]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Creation
complete after 11s
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0727-838d6e2f-5379-493e-b341-d8a9ec75df64]

Refreshing Phase

module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]: Refreshing state...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a]
 
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Refreshing
state...
[id=0727-3479487d-23ad-456c-b274-df0029e4ca6b/0727-838d6e2f-5379-493e-b341-d8a9ec75df64]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Refreshing
state...
[id=0717-d10b9267-e847-462c-9ea3-3ea13ec50790/0717-b538d156-35b1-43b3-9f0c-c102cb3dff6c]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Refreshing
state...
[id=0737-b736bae7-80fa-462e-94f1-a99405c89b24/0737-b374e7c5-29f8-49a8-9da9-8c75a0fbdc98]
 
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Refreshing
state...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0727-838d6e2f-5379-493e-b341-d8a9ec75df64]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Refreshing
state...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0717-b538d156-35b1-43b3-9f0c-c102cb3dff6c]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]:
Refreshing state...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0737-b374e7c5-29f8-49a8-9da9-8c75a0fbdc98]

Will be destroyed

module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]
will be destroyed
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]
will be destroyed
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]
will be destroyed

module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]
will be destroyed

module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]
will be destroyed
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]
will be destroyed
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]
will be destroyed

Destroy Phase

module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]:
Destroying...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0727-838d6e2f-5379-493e-b341-d8a9ec75df64]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Destruction
complete after 1s
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]:
Destroying...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0717-b538d156-35b1-43b3-9f0c-c102cb3dff6c]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Destruction
complete after 1s
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]:
Destroying...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a/0737-b374e7c5-29f8-49a8-9da9-8c75a0fbdc98]
module.vpes.ibm_is_virtual_endpoint_gateway_ip.endpoint_gateway_ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Destruction
complete after 0s
 
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]:
Destroying...
[id=0727-3479487d-23ad-456c-b274-df0029e4ca6b/0727-838d6e2f-5379-493e-b341-d8a9ec75df64]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]:
Destroying...
[id=0737-b736bae7-80fa-462e-94f1-a99405c89b24/0737-b374e7c5-29f8-49a8-9da9-8c75a0fbdc98]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-b-directlink-gateway-2-ip"]: Destruction
complete after 1s
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-c-directlink-gateway-3-ip"]: Destruction
complete after 1s
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]:
Destroying...
[id=0717-d10b9267-e847-462c-9ea3-3ea13ec50790/0717-b538d156-35b1-43b3-9f0c-c102cb3dff6c]
module.vpes.ibm_is_subnet_reserved_ip.ip["vpe-default-2hkz3t-vpc-instance-subnet-a-directlink-gateway-1-ip"]: Destruction
complete after 0s
 
module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]: Destroying...
[id=r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a]
module.vpes.ibm_is_virtual_endpoint_gateway.vpe["vpc-instance-directlink"]: Destruction complete
after 2s

Conclusion:

All the resources which are created are destroyed successfully. I am not sure what does it attempt to destroy further on, hence throw target not found error:

TestRunDefaultExample 2023-05-18T10:30:23Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; ╷
Error: [0m Error: [ERROR] Error getting target(r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a): Target not found
│ {
│     "StatusCode": 404,
│     "Headers": {
│         "Cache-Control": [
│             "max-age=0, no-cache, no-store, must-revalidate"
│         ],
│         "Cf-Cache-Status": [
│             "DYNAMIC"
│         ],
│         "Cf-Ray": [
│             "7c936c22cfe70951-IAD"
│         ],
│         "Content-Type": [
│             "application/json"
│         ],
│         "Date": [
│             "Thu, 18 May 2023 10:30:02 GMT"
│         ],
│         "Expires": [
│             "-1"
│         ],
│         "Pragma": [
│             "no-cache"
│         ],
│         "Server": [
│             "cloudflare"
│         ],
│         "Strict-Transport-Security": [
│             "max-age=31536000; includeSubDomains"
│         ],
│         "Vary": [
│             "Accept-Encoding"
│         ],
│         "X-Content-Type-Options": [
│             "nosniff"
│         ],
│         "X-Request-Id": [
│             "0057c9a8-d8e7-412f-a34b-6b3036565c25"
│         ],
│         "X-Xss-Protection": [
│             "1; mode=block"
│         ]
│     },
│     "Result": {
│         "errors": [
│             {
│                 "code": "not_found",
│                 "message": "Target not found",
│                 "target": {
│                     "name": "id",
│                     "type": "parameter",
│                     "value": "r006-2aaa73bd-dc46-411b-aaf7-98cf3c21007a"
│                 }
│             }
│         ],
│         "trace": "0057c9a8-d8e7-412f-a34b-6b3036565c25"
│     },
│     "RawResult": null
│ }
│ 
│ 
│ 
╵}

@vburckhardt
Copy link
Member

Hello @rajatagarwal-ibm @ocofaigh - where are we on this PR - it seems to have been abandoned?

@ocofaigh
Copy link
Member

ocofaigh commented Jul 20, 2023

@vburckhardt Brought it up with Rajat earlier - he will be handing it over to someone tomorrow to complete since he is off for a few weeks

@rajatagarwal-ibm
Copy link
Member Author

@vburckhardt I'd say we need to create a support case/provider issue to complete this task. I have unassigned myself from the issue for now as I have been going on vacation. If no one else picks that up, I will have a look again.

@rajatagarwal-ibm
Copy link
Member Author

@vburckhardt the above issue was highly impacted by the error "Target not found". There is an issue open for this https://github.ibm.com/GoldenEye/issues/issues/4527

@shemau shemau self-assigned this Jul 31, 2023
@shemau
Copy link
Contributor

shemau commented Aug 2, 2023

/run pipeline

@shemau
Copy link
Contributor

shemau commented Aug 3, 2023

/run pipeline

@shemau
Copy link
Contributor

shemau commented Aug 3, 2023

The upgrade test on PR validation has been skipped due to changes in the test code and new services not being available in the main branch.

Manual tests have been run locally. Checking out main branch, manually deploying examples/default with 3 VPE gateways, running a terraform plan, loading this branch, re-running terraform init --upgrade and then terraform plan again. Both plans identified that the vpe_ips output, which was empty lists, was due to be updated with current IP values. There were no changes to the resources.

@shemau shemau requested a review from Aashiq-J August 3, 2023 09:09
Copy link
Member

@Aashiq-J Aashiq-J left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@ocofaigh ocofaigh merged commit a77b293 into main Aug 3, 2023
2 checks passed
@ocofaigh ocofaigh deleted the add-services-support branch August 3, 2023 11:34
@terraform-ibm-modules-ops
Copy link
Contributor

🎉 This PR is included in version 2.5.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for all services currently supporting VPE
6 participants