Skip to content

xxradar/cilium-clustermesh-multi-region-with-fortigate

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

147 Commits

fgtsingle1

fgtsingle1

 
 

fgtsingle2

fgtsingle2

 
 

images

images

 
 

terraform-k8s-1

terraform-k8s-1

 
 

terraform-k8s-2

terraform-k8s-2

 
 

README.md

README.md

 
 

Repository files navigation

A multi-region Cilium Clustermesh using Fortigate VPN in AWS

Introduction


Setting up the network and fortigate infrastructure

Next terraform plan will create a public and private network and deploy a PAYG fortigate instance.
Rename terraform.tfvars.example to terraform.tfvars and populate the variables.
Deployment will happen in AWS region eu-west-3.

cd fgtsingle1
terraform init
terraform apply

Repeat the process for AWS region eu-west-1 (fgtsingle2)
Give it some minutes. The terraform output will provide login details and IP address of the Fortigates.

Configuring the Fortigates

Create an allow_all outbound rule

Create on both firewall an address resource representing the private networks. - eu-west-3 (fgtsingle1) -> 10.1.1.0/24
- eu-west-1 (fgtsingle2) -> 10.2.1.0/24



Create an outbound firewall rule.




Create a VPN IPSEC tunnel

IPSEC VPN setup using VPN wizard






Deploying Kubernetes and jumpbox

In the next step, a 3-node Kubernetes cluster is deployed, together with a jumpbox and backend test server.
Rename terraform.tfvars.example to terraform.tfvars and populate the variables.
The information can be obtained from the terraform output for both fortigate deployments.

cd terraform-k8s-1
terraform init
terraform apply

Repeat process for terraform-k8s-2.
Wait for a few minutes for all boxes to initialise.

Then run in both terraform-k8s-1 and terraform-k8s-2 folder:

creds.sh 1

Wait for a few minutes for all boxes to initialise.

You can now access the jumpboxes in both regions

ssh -i key0.pem ubuntu@<jumpbox>

From the jumpbox, you can access the k8s master nodes

ssh -i key.pem ubuntu@<k8s_master>

You can always find the IP addresses by running

terraform output

It should look like

backend_private_ip = [
  "10.1.1.203",
]
instancecount = 1
jumbox_public_ip = [
  "15.246.227.203",
]
k8nodes_private_ip = [
  [
    "10.1.1.214",
    "10.1.1.51",
  ],
]
k8smaster_private_ip = [
  "10.1.1.143",
]
private_key = <sensitive>
worker_count = 2

Deploying the clustermesh

Install pre-requisite tooling

On both clusters:

Install required tooling 
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}



export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
curl -L --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-amd64.tar.gz{,.sha256sum}
sha256sum --check hubble-linux-amd64.tar.gz.sha256sum
sudo tar xzvfC hubble-linux-amd64.tar.gz /usr/local/bin
rm hubble-linux-amd64.tar.gz{,.sha256sum}

sudo snap install helm --classic
helm repo add cilium https://helm.cilium.io/

Install Cilium CNI on cluster1

helm install cilium cilium/cilium --version 1.14.2 \
    --namespace kube-system \
    --set authentication.mutual.spire.enabled=true \
    --set authentication.mutual.spire.install.enabled=true \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set encryption.enabled=true \
    --set encryption.type=wireguard \
    --set kube-proxy-replacement=strict \
    --set ingressController.enabled=partial \
    --set ingressController.loadbalancerMode=shared \
    --set ingressController.service.type="NodePort" \
    --set loadBalancer.l7.backend=envoy \
    --set auto-create-cilium-node-resource=true \
    --set ipam.operator.clusterPoolIPv4PodCIDRList="10.10.0.0/16" \
    --set ipam.operator.clusterPoolIPv4MaskSize="24" \
    --set ipam.mode=cluster-pool \
    --set cluster.id=1 \
    --set cluster.name="cluster1"

Install Cilium CNI on cluster2

helm install cilium cilium/cilium --version 1.14.2 \
    --namespace kube-system \
    --set authentication.mutual.spire.enabled=true \
    --set authentication.mutual.spire.install.enabled=true \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set encryption.enabled=true \
    --set encryption.type=wireguard \
    --set kube-proxy-replacement=strict \
    --set ingressController.enabled=partial \
    --set ingressController.loadbalancerMode=shared \
    --set ingressController.service.type="NodePort" \
    --set loadBalancer.l7.backend=envoy \
    --set auto-create-cilium-node-resource=true \
    --set ipam.operator.clusterPoolIPv4PodCIDRList="10.12.0.0/16" \
    --set ipam.operator.clusterPoolIPv4MaskSize="24" \
    --set ipam.mode=cluster-pool \
    --set cluster.id=2 \
    --set cluster.name="cluster2"

Enable the clustermesh service

On both clusters:

cilium clustermesh enable --service-type NodePort

Create a kubeconfig with access to both clusters

Combine both .kube/config files

  • make sure cluster names are different
  • make sure usernames are different
  • make sure context names are different

Follow these steps:
On cluster1

cp .kube/config cluster1.yaml

touch cluster2.yaml

copy/paste .kube/config content (cluster2) into cluster2.yaml
We now have a copy of both kubeconfig files stored in respectively cluster1.yaml and cluster2.yaml

Create unique context, user and cluster references in both files

sed -i 's/kubernetes/kubernetes1/g' cluster1.yaml
sed -i 's/kubernetes/kubernetes2/g' cluster2.yaml

Load the KUBECONFIG

export KUBECONFIG=$PWD/cluster1.yaml:$PWD/cluster2.yaml

~$ kubectl config get-contexts
CURRENT   NAME                           CLUSTER       AUTHINFO            NAMESPACE
*         kubernetes1-admin@kubernetes1   kubernetes1   kubernetes-admin1
          kubernetes2-admin@kubernetes2   kubernetes2   kubernetes-admin2

Connect both clusters

cilium clustermesh connect --context kubernetes1-admin@kubernetes1 --destination-context kubernetes2-admin@kubernetes2

Deploy demo app

Deploy a demo app, based on https://github.com/xxradar/app_routable_demo on both K8S clusters

git clone https://github.com/xxradar/app_routable_demo.git
cd ./app_routable_demo
./setup.sh
watch kubectl get po -n app-routable-demo -o wide

Optionally:

configure Cilium Ingress

kubectl apply -f ./app_routable_demo/ingress_cilium.yaml

Additionally, you need to create a firewall rule allowing access from the internet towards the ingress nodeports.


  • Fortigate provides basic L4 load-balancing through the Virtual server load balance feature
  • Fortigate supports an AWS (and Kubernetes) SDN connector to dynamically populate the K8S nodes.
  • Support for IPS (and other NG features) and basic WAF capabilities.


Testing the clustermesh

App architecture

The ultimate goal is to create following app deployment.



Currently the application is not **globably shared`** accross both clusters.

Connectivity test w/o service sharing

On both clusters, run following test:

kubectl run -it -n app-routable-demo --rm --image xxradar/hackon mycurler -- bash
       curl -v -H "Cookie: loc=client" http://zone1/app3
Result cluster1 
root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.100.23.232:80...
* Connected to zone1 (10.100.23.232) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:44:51 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 600
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"258-zGzwAESu1cvKemzgO8W6mCBaWZQ"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.10.2.91, 10.10.2.183, 10.10.1.240",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.10.2.91",
  "ips": [
    "10.10.2.91",
    "10.10.2.183",
    "10.10.1.240"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-7kztt"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}
Result cluster2 
root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.96.245.46:80...
* Connected to zone1 (10.96.245.46) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:44:49 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 593
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"251-IE7zC2Tif6dhLPEYr0JMi8HMYYM"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.12.2.8, 10.12.0.40, 10.12.2.16",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.12.2.8",
  "ips": [
    "10.12.2.8",
    "10.12.0.40",
    "10.12.2.16"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-fqh25"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.96.245.46:80...
* Connected to zone1 (10.96.245.46) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:44:49 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 593
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"251-IE7zC2Tif6dhLPEYr0JMi8HMYYM"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.12.2.8, 10.12.0.40, 10.12.2.16",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.12.2.8",
  "ips": [
    "10.12.2.8",
    "10.12.0.40",
    "10.12.2.16"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-fqh25"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}

Connectivity test global service sharing

Annotate a service (zone1)

  annotations:
    service.cilium.io/global: "true"
    service.cilium.io/shared: "true"

You can test this:

kubectl run -it -n app-routable-demo --rm --image xxradar/hackon mycurler -- bash
       curl -v -H "Cookie: loc=client" http://zone1/app3
Result cluster1 
root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.100.210.61:80...
* Connected to zone1 (10.100.210.61) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:36:10 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 599
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"257-8F7QuYIPBkqxghJh0J7OdGFUAJY"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.10.2.159, 10.12.0.40, 10.12.2.16",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.10.2.159",
  "ips": [
    "10.10.2.159",
    "10.12.0.40",
    "10.12.2.16"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-f754s"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}
Result cluster2 
root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.98.218.19:80...
* Connected to zone1 (10.98.218.19) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:38:01 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 603
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"25b-Sh6Qc2jjVtVZLUmH1G3UFezpX0E"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.12.2.122, 10.10.2.183, 10.10.1.240",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.12.2.122",
  "ips": [
    "10.12.2.122",
    "10.10.2.183",
    "10.10.1.240"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-7kztt"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}

Disable sharing on cluster2

Annotate a service (zone1) (cluster2)

  annotations:
    service.cilium.io/global: "true"
    service.cilium.io/shared: "false"

You can test this:

kubectl run -it -n app-routable-demo --rm --image xxradar/hackon mycurler -- bash
       curl -v -H "Cookie: loc=client" http://zone1/app3
Result cluster1 
root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.100.210.61:80...
* Connected to zone1 (10.100.210.61) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:29:07 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 603
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"25b-f0L/f1mAbIn+H04wKxbGvqLshvs"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.10.2.159, 10.10.2.183, 10.10.1.240",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.10.2.159",
  "ips": [
    "10.10.2.159",
    "10.10.2.183",
    "10.10.1.240"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-l6z8f"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}
Result cluster2 
root@mycurler:/# curl -v -H "Cookie: loc=client" http://zone1/app3
*   Trying 10.98.218.19:80...
* Connected to zone1 (10.98.218.19) port 80 (#0)
> GET /app3 HTTP/1.1
> Host: zone1
> User-Agent: curl/7.81.0
> Accept: */*
> Cookie: loc=client
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.25.2
< Date: Tue, 19 Sep 2023 10:30:47 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 603
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"25b-NpEDfnU4V3ztZSYebd9GcZlJZOU"
<
{
  "path": "/app3",
  "headers": {
    "x-forwarded-for": "10.12.2.137, 10.10.2.183, 10.10.1.240",
    "cookie": "loc=client, loc=zone1, loc=zone3, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "10.12.2.137",
  "ips": [
    "10.12.2.137",
    "10.10.2.183",
    "10.10.1.240"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-7kztt"
  },
  "connection": {}
* Connection #0 to host zone1 left intact
}

Ingress + GSLB testing


Ingress results

fgtsingle1_eip

% curl 35.180.16.215/app2
{
  "path": "/app2",
  "headers": {
    "x-forwarded-for": "84.192.8.14, 10.10.2.138, 10.10.2.183, 10.10.1.22",
    "cookie": ", loc=zone1, loc=zone2, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/8.1.2",
    "accept": "*/*",
    "x-forwarded-proto": "http",
    "x-envoy-external-address": "84.192.8.14",
    "x-request-id": "7cb335aa-a6b5-43ac-aa33-265220a46259"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "84.192.8.14",
  "ips": [
    "84.192.8.14",
    "10.10.2.138",
    "10.10.2.183",
    "10.10.1.22"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-x5pkw"
  },
  "connection": {}
}

fgtsingle2_eip

% curl 34.242.99.90/app1
{
  "path": "/app1",
  "headers": {
    "x-forwarded-for": "84.192.8.14, 10.12.2.54, 10.12.0.40, 10.12.0.239",
    "cookie": ", loc=zone1, loc=zone2, loc=zone4",
    "host": "zone6",
    "connection": "close",
    "user-agent": "curl/8.1.2",
    "accept": "*/*",
    "x-forwarded-proto": "http",
    "x-envoy-external-address": "84.192.8.14",
    "x-request-id": "b45d921d-a2e4-4a98-963e-04cf2e345ee5"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone6",
  "ip": "84.192.8.14",
  "ips": [
    "84.192.8.14",
    "10.12.2.54",
    "10.12.0.40",
    "10.12.0.239"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-1-deployment-695c7db8d-ljnw6"
  },
  "connection": {}
}

Annotations

Annotate a service (zone1) (cluster1)

  annotations:
    service.cilium.io/global: "true"
    service.cilium.io/shared: "true"

Annotate a service (zone1) (cluster2)

  annotations:
    service.cilium.io/global: "true"
    service.cilium.io/shared: "false"

fgtsingle1_eip

% curl 35.180.16.215/app2
{
  "path": "/app2",
  "headers": {
    "x-forwarded-for": "84.192.8.14, 10.10.2.138, 10.12.0.40, 10.12.0.239",
    "cookie": ", loc=zone1, loc=zone2, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/8.1.2",
    "accept": "*/*",
    "x-forwarded-proto": "http",
    "x-envoy-external-address": "84.192.8.14",
    "x-request-id": "65deeed9-0c1b-406a-b078-69de93451ef2"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "84.192.8.14",
  "ips": [
    "84.192.8.14",
    "10.10.2.138",
    "10.12.0.40",
    "10.12.0.239"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-fqh25"
  },
  "connection": {}
}

fgtsingle2_eip

 curl 34.242.99.90/app1
{
  "path": "/app1",
  "headers": {
    "x-forwarded-for": "84.192.8.14, 10.12.2.54, 10.12.0.40, 10.12.0.239",
    "cookie": ", loc=zone1, loc=zone2, loc=zone4",
    "host": "zone6",
    "connection": "close",
    "user-agent": "curl/8.1.2",
    "accept": "*/*",
    "x-forwarded-proto": "http",
    "x-envoy-external-address": "84.192.8.14",
    "x-request-id": "c7425a21-cc19-4d45-b64e-05e8ffe7323d"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone6",
  "ip": "84.192.8.14",
  "ips": [
    "84.192.8.14",
    "10.12.2.54",
    "10.12.0.40",
    "10.12.0.239"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-1-deployment-695c7db8d-ljnw6"
  },
  "connection": {}
}
GSLB results

AMS DC client (should resolve to eu-west-3)

root@ubuntu-s-2vcpu-4gb-amd-ams3-01:~# curl app-demo.kubiosec.tech/app2
{
  "path": "/app2",
  "headers": {
    "x-forwarded-for": "167.99.41.154, 10.10.2.138, 10.12.0.40, 10.12.0.239",
    "cookie": ", loc=zone1, loc=zone2, loc=zone5",
    "host": "zone7",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*",
    "x-forwarded-proto": "http",
    "x-envoy-external-address": "167.99.41.154",
    "x-request-id": "e921a999-83d1-4abe-af4c-cf2d26c8545f"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone7",
  "ip": "167.99.41.154",
  "ips": [
    "167.99.41.154",
    "10.10.2.138",
    "10.12.0.40",
    "10.12.0.239"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-2-deployment-6f499cfbbb-fqh25"
  },
  "connection": {}
}

NY DC client (should resolve to eu-west-1)

root@ubuntu-s-2vcpu-4gb-amd-nyc3-01:~# curl http://app-demo.kubiosec.tech/app1
{
  "path": "/app1",
  "headers": {
    "x-forwarded-for": "174.138.84.160, 10.12.2.54, 10.12.0.40, 10.12.0.239",
    "cookie": ", loc=zone1, loc=zone2, loc=zone4",
    "host": "zone6",
    "connection": "close",
    "user-agent": "curl/7.81.0",
    "accept": "*/*",
    "x-forwarded-proto": "http",
    "x-envoy-external-address": "174.138.84.160",
    "x-request-id": "50e838d4-72a5-411d-93fc-84a0113d1d3c"
  },
  "method": "GET",
  "body": "",
  "fresh": false,
  "hostname": "zone6",
  "ip": "174.138.84.160",
  "ips": [
    "174.138.84.160",
    "10.12.2.54",
    "10.12.0.40",
    "10.12.0.239"
  ],
  "protocol": "http",
  "query": {},
  "subdomains": [],
  "xhr": false,
  "os": {
    "hostname": "echoserver-1-deployment-695c7db8d-hbnw6"
  },
  "connection": {}
}
Latency




About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

4 stars

Watchers

1 watching

Forks

2 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages