Test https-hosts as part of diagnose

[kylos101]

Description

This helps customers test outbound HTTPS connectivity from the main subnet. This helps assert connectivity (or not) for SSO and VCS integrations.

Related Issue(s)

Fixes ENT-630

How to test

From a private cell, w/o adding a route to send 0.0.0.0/0 over the transit gateway, run diagnose. Then add rule to the main routetable for the Gitpod VPC, and try again. It should fail the first time, and succeed the second time.

Happy path (a private cell that has a transit gateway attached, where the main subnets route 0.0.0.0/0 to it):

go run . diagnose
INFO[0000] ✅ Main Subnets are valid                     
INFO[0000] ✅ Pod Subnets are valid                      
INFO[0000] ℹ️  Checking prerequisites                   
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.ec2messages is configured 
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.ssm is configured 
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.ssmmessages is configured 
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.execute-api is configured 
INFO[0000] ✅ IAM role created and policy attached       
INFO[0001] ℹ️  Launching EC2 instances in Main subnets  
INFO[0001] ℹ️  Created security group with ID: sg-06376cbb96e71a82a 
INFO[0008] ℹ️  Created security group with ID: sg-036424fa9cb7d9c05 
INFO[0010] ℹ️  Main EC2 instances: [i-060e01f1e2a49cbb2 i-0a530c569bf0bdd22] 
INFO[0010] ℹ️  Launching EC2 instances in a Pod subnets 
INFO[0011] ℹ️  Created security group with ID: sg-042a774ed939bd6ea 
INFO[0013] ℹ️  Created security group with ID: sg-084a2f9a55d3de63c 
INFO[0015] ℹ️  Pod EC2 instances: [i-04add4a0733995527 i-012c4e9c46a70dd2f] 
INFO[0015] ℹ️  Waiting for EC2 instances to become ready (can take up to 2 minutes) 
INFO[0043] ✅ EC2 Instances are now running successfully 
INFO[0043] ℹ️  Connecting to SSM...                     
INFO[0118] ℹ️  Checking if the required AWS Services can be reached from the ec2 instances 
INFO[0119] ✅ Autoscaling is available                   
INFO[0120] ✅ CloudFormation is available                
INFO[0121] ✅ CloudWatch is available                    
INFO[0122] ✅ EC2 is available                           
INFO[0123] ✅ EC2messages is available                   
INFO[0124] ✅ ECR is available                           
INFO[0126] ✅ ECR Api is available                       
INFO[0127] ✅ EKS is available                           
INFO[0128] ✅ Elastic LoadBalancing is available         
INFO[0129] ✅ KMS is available                           
INFO[0130] ✅ Kinesis Firehose is available              
INFO[0131] ✅ SSM is available                           
INFO[0132] ✅ SSMmessages is available                   
INFO[0134] ✅ SecretsManager is available                
INFO[0135] ✅ Sts is available                           
INFO[0135] ℹ️  Checking if certain AWS Services can be reached from ec2 instances in the main subnet 
INFO[0135] ✅ DynamoDB is available                      
INFO[0136] ✅ S3 is available                            
INFO[0136] ℹ️  Checking if hosts can be reached with HTTPS from ec2 instances in the main subnets 
INFO[0138] ✅ accounts.google.com is available           
INFO[0139] ✅ github.com is available                    
INFO[0139] ✅ Instances terminated                       
INFO[0139] Cleaning up: Waiting for 2 minutes so network interfaces are deleted 

Sad path (I removed the routes to the transit gateway from each route table for the main subnets):

go run . diagnose
INFO[0000] ✅ Main Subnets are valid                     
INFO[0000] ✅ Pod Subnets are valid                      
INFO[0000] ℹ️  Checking prerequisites                   
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.ec2messages is configured 
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.ssm is configured 
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.ssmmessages is configured 
INFO[0000] ✅ VPC endpoint com.amazonaws.eu-central-1.execute-api is configured 
INFO[0000] ✅ IAM role created and policy attached       
INFO[0001] ℹ️  Launching EC2 instances in Main subnets  
INFO[0001] ℹ️  Created security group with ID: sg-083e624ef542d6440 
INFO[0008] ℹ️  Created security group with ID: sg-08e6f4a3f50dafdc0 
INFO[0009] ℹ️  Main EC2 instances: [i-09995627fb62cf05c i-0f1860b2d8c8be1e4] 
INFO[0009] ℹ️  Launching EC2 instances in a Pod subnets 
INFO[0010] ℹ️  Created security group with ID: sg-0d3dc26f32cabc950 
INFO[0012] ℹ️  Created security group with ID: sg-02d166502041318b3 
INFO[0014] ℹ️  Pod EC2 instances: [i-08bd3bc780f75c3d6 i-0f6608036eb51ce4a] 
INFO[0014] ℹ️  Waiting for EC2 instances to become ready (can take up to 2 minutes) 
INFO[0045] ✅ EC2 Instances are now running successfully 
INFO[0045] ℹ️  Connecting to SSM...                     
INFO[0123] ℹ️  Checking if the required AWS Services can be reached from the ec2 instances 
INFO[0125] ✅ Autoscaling is available                   
INFO[0126] ✅ CloudFormation is available                
INFO[0127] ✅ CloudWatch is available                    
INFO[0128] ✅ EC2 is available                           
INFO[0129] ✅ EC2messages is available                   
INFO[0130] ✅ ECR is available                           
INFO[0131] ✅ ECR Api is available                       
INFO[0132] ✅ EKS is available                           
INFO[0134] ✅ Elastic LoadBalancing is available         
INFO[0135] ✅ KMS is available                           
INFO[0136] ✅ Kinesis Firehose is available              
INFO[0137] ✅ SSM is available                           
INFO[0138] ✅ SSMmessages is available                   
INFO[0139] ✅ SecretsManager is available                
INFO[0140] ✅ Sts is available                           
INFO[0140] ℹ️  Checking if certain AWS Services can be reached from ec2 instances in the main subnet 
INFO[0141] ✅ DynamoDB is available                      
INFO[0142] ✅ S3 is available                            
INFO[0142] ℹ️  Checking if hosts can be reached with HTTPS from ec2 instances in the main subnets 
WARN[0157] ❌ accounts.google.com is not available (https://accounts.google.com) 
INFO[0157] ❌ Error fetching command results: instance i-0f1860b2d8c8be1e4 command failed:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
curl: (28) Connection timed out after 15000 milliseconds
failed to run commands: exit status 28 
WARN[0173] ❌ github.com is not available (https://github.com) 
INFO[0173] ❌ Error fetching command results: instance i-0f1860b2d8c8be1e4 command failed:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
curl: (28) Connection timed out after 15001 milliseconds
failed to run commands: exit status 28 
INFO[0173] ✅ Instances terminated                       
INFO[0173] Cleaning up: Waiting for 2 minutes so network interfaces are deleted 

Documentation

/hold