Apigee hybrid on GKE

.gitignore

@@ -45,3 +45,7 @@ blueprints/apigee/bigquery-analytics/deploy-apiproxy.sh
 blueprints/apigee/network-patterns/nb-glb-psc-neg-sb-psc-ilbl7-hybrid-neg/bundle/apiproxy/targets/default.xml
 blueprints/apigee/network-patterns/nb-glb-psc-neg-sb-psc-ilbl7-hybrid-neg/bundle.zip
 blueprints/apigee/network-patterns/nb-glb-psc-neg-sb-psc-ilbl7-hybrid-neg/deploy-apiproxy.sh
+blueprints/apigee/hybrid-gke/apiproxy.zip
+blueprints/apigee/hybrid-gke/deploy-apiproxy.sh
+blueprints/apigee/hybrid-gke/ansible/gssh.sh
+blueprints/apigee/hybrid-gke/ansible/vars/vars.yaml

blueprints/apigee/README.md

@@ -1,7 +1,8 @@
-# Apigee Examples
+# Apigee Blueprints
 
 This repository contains the following Apigee examples:
 
+* [Apigee Hybrid on GKE](./hybrid-gke/README.md)
 * [Apigee BigQuery analytics](./bigquery-analytics/README.md)
 * Apigee network patterns
     * [Apigee X - Northbound GLB with PSC Neg, Southbouth PSC with ILB (L7) and Hybrid NEG

blueprints/apigee/hybrid-gke/README.md

@@ -0,0 +1,59 @@
+# Apigee Hybrid on GKE
+
+This example installs Apigee hybrid in a non-prod environment on a GKE private cluster using Terraform and Ansible.
+The Terraform configuration deploys all the required infrastructure including a management VM used to run an ansible playbook to the actual Apigee Hybrid setup.
+
+The diagram below depicts the architecture.
+
+![Diagram](./diagram.png)
+
+## Running the blueprint
+
+1. Clone this repository or [open it in cloud shell](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fcloud-foundation-fabric&cloudshell_print=cloud-shell-readme.txt&cloudshell_working_dir=blueprints%2Fapigee%2Fhybrid), then go through the following steps to create resources:
+
+2. Copy the file [terraform.tfvars.sample](./terraform.tfvars.sample) to a file called ```terraform.tfvars``` and update the values if required.
+
+3. Initialize the terraform configuration
+
+    ```
+    terraform init
+    ```
+
+4. Apply the terraform configuration
+
+    ```
+    terraform apply
+    ```
+
+## Testing the blueprint
+
+2. Deploy an api proxy
+
+    ```
+    ./deploy-apiproxy.sh
+    ```
+
+3. In the console check the IP address that has been allocated to the Apigee ingress gateway and send some traffic to the deployed API proxy.
+
+    ```
+    curl -k -v -H "Host:HOSTNAME" \
+    --resolve HOSTNAME:443:IP_ADDRESS \
+    https://HOSTNAME/httpbin/headers
+    ```
+<!-- BEGIN TFDOC -->
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [hostname](variables.tf#L43) | Host name. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L79) | Project ID. | <code>string</code> | ✓ |  |
+| [cluster_machine_type](variables.tf#L17) | Cluster nachine type. | <code>string</code> |  | <code>&#34;e2-standard-4&#34;</code> |
+| [cluster_network_config](variables.tf#L23) | Cluster network configuration. | <code title="object&#40;&#123;&#10;  nodes_cidr_block              &#61; string&#10;  pods_cidr_block               &#61; string&#10;  services_cidr_block           &#61; string&#10;  master_authorized_cidr_blocks &#61; map&#40;string&#41;&#10;  master_cidr_block             &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  nodes_cidr_block    &#61; &#34;10.0.1.0&#47;24&#34;&#10;  pods_cidr_block     &#61; &#34;172.16.0.0&#47;20&#34;&#10;  services_cidr_block &#61; &#34;192.168.0.0&#47;24&#34;&#10;  master_authorized_cidr_blocks &#61; &#123;&#10;    internal &#61; &#34;10.0.0.0&#47;8&#34;&#10;  &#125;&#10;  master_cidr_block &#61; &#34;10.0.0.0&#47;28&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
+| [mgmt_server_config](variables.tf#L48) | Mgmt server configuration. | <code title="object&#40;&#123;&#10;  disk_size     &#61; number&#10;  disk_type     &#61; string&#10;  image         &#61; string&#10;  instance_type &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  disk_size     &#61; 50&#10;  disk_type     &#61; &#34;pd-ssd&#34;&#10;  image         &#61; &#34;projects&#47;ubuntu-os-cloud&#47;global&#47;images&#47;family&#47;ubuntu-2204-lts&#34;&#10;  instance_type &#61; &#34;n1-standard-2&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
+| [mgmt_subnet_cidr_block](variables.tf#L64) | Management subnet CIDR block. | <code>string</code> |  | <code>&#34;10.0.2.0&#47;28&#34;</code> |
+| [project_create](variables.tf#L70) | Parameters for the creation of the new project. | <code title="object&#40;&#123;&#10;  billing_account_id &#61; string&#10;  parent             &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [region](variables.tf#L84) | Region. | <code>string</code> |  | <code>&#34;europe-west1&#34;</code> |
+| [zone](variables.tf#L90) | Zone. | <code>string</code> |  | <code>&#34;europe-west1-c&#34;</code> |
+
+<!-- END TFDOC -->

blueprints/apigee/hybrid-gke/ansible.tf

@@ -0,0 +1,39 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Ansible generated files.
+
+resource "local_file" "vars_file" {
+  content = yamlencode({
+    cluster    = module.cluster.name
+    region     = var.region
+    project_id = module.project.project_id
+    envgroup   = local.envgroup
+    env        = local.environment
+    hostname   = var.hostname
+  })
+  filename        = "${path.module}/ansible/vars/vars.yaml"
+  file_permission = "0666"
+}
+
+resource "local_file" "gssh_file" {
+  content = templatefile("${path.module}/templates/gssh.sh.tpl", {
+    project_id = module.project.project_id
+    zone       = var.zone
+  })
+  filename        = "${path.module}/ansible/gssh.sh"
+  file_permission = "0777"
+}

blueprints/apigee/hybrid-gke/ansible/ansible.cfg

@@ -0,0 +1,8 @@
+[defaults]
+inventory = inventory/hosts.ini
+timeout = 900
+
+[ssh_connection]
+pipelining = True
+ssh_executable = ./gssh.sh
+transfer_method = piped

blueprints/apigee/hybrid-gke/ansible/inventory/hosts.ini

@@ -0,0 +1 @@
+mgmt

blueprints/apigee/hybrid-gke/ansible/playbook.yaml

@@ -0,0 +1,26 @@
+# Copyright 2022 Google LLC
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- hosts: mgmt
+  gather_facts: "no"
+  vars_files:
+    - vars/vars.yaml
+  environment:
+    USE_GKE_GCLOUD_AUTH_PLUGIN: True
+  roles:
+    - role: prerequisites
+      become: yes
+      become_method: sudo
+    - role: apigee-hybrid
+

blueprints/apigee/hybrid-gke/ansible/roles/apigee-hybrid/tasks/main.yaml

@@ -0,0 +1,143 @@
+# Copyright 2023 Google LLC
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- name: Get cluster credentials
+  shell: >
+    gcloud container clusters get-credentials {{ cluster }} \
+    --region {{ region }} \
+    --project {{ project_id }} \
+    --internal-ip
+
+- name: Install cert-manager
+  shell: >
+    kubectl apply \
+    --validate=false \
+    -f https://github.com/jetstack/cert-manager/releases/download/v1.7.2/cert-manager.yaml
+
+- name: Wait until pods are ready in cert-manager namespace
+  shell: >
+    kubectl wait --for=condition=ready pods \
+    -l app.kubernetes.io/instance=cert-manager \
+    -n cert-manager \ 
+    --timeout=90s
+
+- name: Fetch apigeectl version
+  uri:
+    url: https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/current-version.txt?ignoreCache=1
+    return_content: yes
+  register: version
+
+- name: Download apigeectl bundle
+  uri:
+    url: https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/{{ version.content }}/apigeectl_linux_64.tar.gz
+    dest: "~/apigeectl.tar.gz"
+    status_code: [200, 304]
+
+- name: Extract apigeectl bundle
+  unarchive:
+    src: "~/apigeectl.tar.gz"
+    dest: "~"
+    remote_src: yes    
+
+- name: Move apigeectl folder
+  shell: >
+    mv ~/apigeectl_* ~/apigeectl
+
+- name: Create hybrid-files
+  file:
+    path: "~/hybrid-files/{{ item }}"
+    state: directory
+  with_items:
+    - overrides
+    - certs
+
+- name: Create a symbolic links
+  file:
+    src: ~/apigeectl/{{ item }}
+    dest: "~/hybrid-files/{{ item }}"
+    state: link    
+  with_items:
+    - tools
+    - config
+    - templates
+    - plugins    
+
+- name: Create service accounts
+  shell: >
+    ~/hybrid-files/tools/create-service-account -i {{ project_id }} -e non-prod -d ~/hybrid-files/service-accounts
+
+- name: Create certificates
+  shell: >
+    openssl req  \ 
+    -nodes \
+    -new \
+    -x509 \
+    -keyout ~/hybrid-files/certs/{{ envgroup }}.key \
+    -out ~/hybrid-files/certs/{{ envgroup }}.cert -subj '/CN='{{ hostname }}'' -days 3650
+
+- name: Create overrides.yaml
+  template:
+    src: templates/overrides.yaml.j2
+    dest: ~/hybrid-files/overrides/overrides.yaml
+
+- name: Enable syncronizer access
+  shell: >
+    curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \
+    -H "Content-Type:application/json" \
+    "https://apigee.googleapis.com/v1/organizations/{{ project_id }}:setSyncAuthorization" \
+    -d '{"identities":["'"serviceAccount:apigee-non-prod@{{ project_id }}.iam.gserviceaccount.com"'"]}'
+
+- name: Dry-run (init)
+  shell: >
+    ~/apigeectl/apigeectl init -f overrides/overrides.yaml --dry-run=client  
+  args:
+    chdir: ~/hybrid-files
+
+- name: Install the Apigee deployment services Apigee Deployment Controller and Apigee Admission Webhook.
+  shell: >
+    ~/apigeectl/apigeectl init -f overrides/overrides.yaml 
+  args:
+    chdir: ~/hybrid-files    
+
+- name: Wait until pods are ready in apigee-system namespace
+  shell: >
+    kubectl wait --for=condition=ready pods \
+    -l app=apigee-controller \
+    -n apigee-system \ 
+    --timeout=300s    
+
+- name: Wait until pods are ready in apigee namespace
+  shell: >
+    kubectl wait --for=condition=ready pods \
+    -l app=apigee-ingressgateway-manager \
+    -n apigee \ 
+    --timeout=300s    
+
+- name: Dry-run (apply)
+  shell: >
+    ~/apigeectl/apigeectl apply -f overrides/overrides.yaml --dry-run=client  
+  args:
+    chdir: ~/hybrid-files
+
+- name: Install the Apigee runtime components
+  shell: >
+    ~/apigeectl/apigeectl apply -f overrides/overrides.yaml 
+  args:
+    chdir: ~/hybrid-files        
+
+- name: Check status of the deployment
+  shell: >
+    while [ -n "$(kubectl get pods -n apigee | tail -n +2 | grep -v Running | grep -v Completed)" ]; do sleep 1; done
+  args:
+    chdir: ~/hybrid-files

blueprints/apigee/hybrid-gke/ansible/roles/apigee-hybrid/templates/overrides.yaml.j2

@@ -0,0 +1,63 @@
+gcp:
+  region: {{ region }}
+  projectID: {{ project_id }}
+
+k8sCluster:
+  name: {{ cluster }}
+  region: CLUSTER_LOCATION # Must be the closest Google Cloud region to your cluster.
+org: {{ project_id }}
+
+instanceID: "instance-1"
+
+cassandra:
+  hostNetwork: false
+    # Set to false for single region installations and multi-region installations
+    # with connectivity between pods in different clusters, for example GKE installations.
+    # Set to true  for multi-region installations with no communication between
+    # pods in different clusters, for example GKE On-prem, GKE on AWS, Anthos on bare metal,
+    # AKS, EKS, and OpenShift installations.
+    # See Multi-region deployment: Prerequisites
+
+virtualhosts:
+  - name: {{ envgroup }}
+    selector:
+      app: apigee-ingressgateway
+    sslCertPath: ./certs/{{ envgroup }}.cert
+    sslKeyPath: ./certs/{{ envgroup }}.key
+
+ao:
+  args:
+  # This configuration is introduced in hybrid v1.8
+    disableIstioConfigInAPIServer: true
+
+# This configuration is introduced in hybrid v1.8
+ingressGateways:
+- name: ingress # maximum 17 characters. See Known issue 243167389.
+  replicaCountMin: 2
+  replicaCountMax: 10
+
+envs:
+  - name: {{ env }}
+    serviceAccountPaths:
+      synchronizer: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+      udca: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+      runtime: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+mart:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+connectAgent:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+metrics:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+udca:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+watcher:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+logger:
+  enabled: true
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json