-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ground to Cloud enablement through PSC (private service connect) or PGA (private google access) through an interconnect or VPN for private GCP API access - customer procedure using AWS as simulated groud #299
Comments
AWS (prem/ground) to GCP HA VPN up for private google access and DNS forwarding simulation in |
Shadow Procedure: GCPCreate projects - GCP
Set iAM permissions
bug: routing mode should be regional
create network
create HA VPN Gateway
AWSCreate 2 customer gateways
Create VPC Bug: https://cloud.google.com/network-connectivity/docs/vpn/tutorials/create-ha-vpn-connections-google-cloud-aws#create_gateways_and_vpn_connections_on_aws Create AWS VPC
BUG: VPC already has a route table with a default route after vpc creation BUG: switch subnet to us-east-1a (to match VPC) - or VPC missing region
BUG: VPC must be /16 not /24 - in order to use subnets of /20
create subnet
Recreate a new VPC
Add subnet
No AWS IGW or NGW - but put a NGW for the private subnet - step 6
Create VM
finished with Attach VPG to VPC
follow for options generate preshared keys
BUG: shared key site generates / and + (invalid chars) - convert to .
VPNs take a couple min to transition from pending step GCP 2 - VPN Tunnels
4 tunnels use ike-version=2
aws side - IP_SEC is up only so far until we setup BGP 20230827 Add 3 more tunnel
takes about 4 min for all 4 4 router interfaces Get IPs from AWS VPN config (generic IKE2) - the customer gateway address Inside IP Addresses
4 add BGP peers --peer-ip-address: invalid ipv4 value: '169.254.51.0/30' bug: CIDR not recognized - need IP like 169.254.51.1 from the vpn config - the BGP neighbor IP
in another working VPN of my we use 169.254.0.2 - this is the GOOGLE_BGP_IP_TUNNEL_1 address
Add 3 remaining BGP sessions
1007-1011 - 4 min for AWS side
last of 4 bgp tunnels
All 4 tunnels up on both sides check routes use a bastion Verify BGP dynamic routes on both GCP and AWS sides
|
Add public subnet and bastion for VM access on AWS attach IGW to VPC
remove NATGW from private subnet - not needed - move to public subnet - remove black hole create new NATGW with existing EIP add route to IGW from public subnet
Been a while IGW and NAT dont' have route table entries - adding 0.0.0.0/0 to IGW regular ssh working now
Tunnel via public subnet bastion through private subnet VM - in AWS
Test IP Private Google Access before adding a private googleapis.com zone in route53
Add private.googleapis.com custom route to router on GCP sideFix nat on private subnet on aws - prior to viewing bgp dynamic routes from gcp
Add PGA route in AWS on private subnet not by pointing to the IGW but my advertising a custom route to 199.36.153.8/30 that is picked up by the VPN on the AWS side's BGP router Check Routes
We can see the private google access CIDR in the first advertised route along with the private subnet in GCP VPC
We should not need to - as it does not affect the on-prem network - but turn on PGA for the GCP private subnet Test connectivity back to AWS using a VM in GCP
however we see AWS routes in GCP - but only the VPC - which may be subnet/vpc routing on the AWS side
first verify connectivity between VMs in both CSPs - spin up 2 more VMs
Checking AWS propagation on the routes - off - this may be the issue editing route propagation to route through the VPG Routing working now from GCP to AWS
we can ping from GCP to AWS (prem) and we can ping the reverse AWS(prem) to GCP
|
Switch from IP to DNS resolution for private google access from AWS to GCP
add private DNS zone on prem (AWS) for private.googleapis.com in Route53https://us-east-1.console.aws.amazon.com/route53/v2/hostedzones?region=us-east-1#CreateHostedZone Plural A records use CR/LF separators
For each VPC that you associate with a private hosted zone, you must set the Amazon VPC settings enableDnsHostnames and enableDnsSupport to true. https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html#vpc-dns-updating The "enable dns hostnames" was not set - we don't want it set - just "resolution" should be set Do a dig on the private VM inside AWS
private domain not having effect on the VPC - checking dhcp options Checking traceroute
check /etc/hosts.txt override
|
Verify GCP API calls via gcloud CLI on AWSInstall gcloud cli
How do I verify that googleapis.com traffic is not going through the NAT or IGW on the public subnet - check routes
Destination Add a GCS bucket to be able to list/update from AWS from AWS
|
Update: PGA is working as expected from on-prem (simulated by an AWS VPC via VPN - which works well as a non-GCP on-prem/ground) following procedures in the github issue below. Essentially the main changes are what is detailed in the deck and docs. Verified 1 and 2 GoogleCloudPlatform/pubsec-declarative-toolkit#494 on an AWS private VM (ping back to GCP VM, run a GCS ls) |
changes
PSC need to enable
See related forwarding rule issue after servicedirectory enablement - we get the service directory entry - although with a default region - to be fixed Verify static internal IP for the PSC endpoint switch from the global google_compute_global_forwarding_rule to google_compute_forwarding_rule https://github.com/hashicorp/terraform-provider-google-beta/blob/main/website/docs/r/compute_forwarding_rule.html.markdown For regional change to the PSC endpoint
|
#299 - Initial version of private service connect - with global endpoint
pending
|
PSC IP added to router advertisements - test results for PR
|
#299 - add PSC IP to router advertisements
Test results for DNS ingress policy for PSC endpoint
|
#299 - add DNS Server Policy - DNS egress proxy for PSC and generated static IP
Testing within gcloud
testing on prem |
By default global dynamic routing on the vpc is on/global to accommodate PSC mode global over regional |
Add service account for bigquery
see
test access by temporarily deleting bigquery.admin role Reauthenticate impersonation to check biqquery deny
Reinstate bigquery.admin
|
20240406: Closing issue during retrofit/rebase of this TEF V1 based/modified repo to TEF V4 standards |
shadow #286
and
GoogleCloudPlatform/pubsec-declarative-toolkit#468
20230827:2300: pivot to PSC from PGA
https://cloud.google.com/vpc/docs/about-accessing-vpc-hosted-services-endpoints
https://cloud.google.com/vpc/docs/configure-private-service-connect-apis
Document and simulate GCP + Customer procedure:
shadow
GoogleCloudPlatform/pubsec-declarative-toolkit#494
follow
https://cloud.google.com/vpc/docs/private-access-options
Requirements
Asset Inventory - GCP side
Notes:
https://docs.google.com/presentation/d/13sjT2tJ4yLIYGRREE3wBrylB1OvcEMpKdquVuJB_nX4/edit?resourcekey=0-N3DruQaiutFvZ98HTT7-vQ#slide=id.g1154b3b950f_2_3458
slide 27
https://cloud.google.com/vpc/docs/configure-private-google-access-hybrid
https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#on-premises
Customer environment consists of already created interconnect/VPN where there is a BGP route for the DNS proxy egress from onprem
Use case is one where google APIs and googledomains.com queries into GCP both resolve and are kept private on the premium google network
https://cloud.google.com/vpc/docs/private-service-connect#:~:text=Similarly%2C%20a%20Private%20Service%20Connect,internal%20IP%20addresses%20for%20endpoints.
Reference: procedures
The text was updated successfully, but these errors were encountered: