Merge commit '7779a97268893f69a20be761e8545270fa601dc7' into lukeshu/…

…2.3.2
telepresenceio · Jun 18, 2021 · 80e2943 · 80e2943
2 parents 263e700 + b9722fe
commit 80e2943
Show file tree

Hide file tree

Showing 11 changed files with 222 additions and 4 deletions.
diff --git a/doc-links.yml b/doc-links.yml
@@ -26,6 +26,8 @@
        link: howtos/preview-urls
      - title: Proxy outbound traffic to my cluster
        link: howtos/outbound
+     - title: Send requests to an intercepted service
+       link: howtos/request
  - title: Technical reference
    items:
      - title: Architecture

diff --git a/faqs.md b/faqs.md
@@ -61,6 +61,14 @@ You can connect to databases or middleware running in the cluster, such as MySQL
 
 Telepresence will discover/prompt during first use for this info and make its best guess at figuring this out and ask you to confirm or update this.
 
+** Why are my intercepts still reporting as active when they've been disconnected?**
+
+  In certain cases, Telepresence might not have been able to communicate back with Ambassador Cloud to update the intercept's status. Worry not, they will get garbage collected after a period of time.
+
+** Why is my intercept associated with an "Unreported" cluster?**
+
+  Intercepts tagged with "Unreported" clusters simply mean Ambassador Cloud was unable to associate a service instance with a known detailed service from an Edge Stack or API Gateway cluster. [Connecting your cluster to the Service Catalog](../../service-catalog/quick-start/) will properly match your services from multiple data sources.
+
 ** Will Telepresence be able to intercept workloads running on a private cluster or cluster running within a virtual private cloud (VPC)?**
 
  Yes. The cluster has to have outbound access to the internet for the preview URLs to function correctly, but it doesn’t need to have a publicly accessible IP address.

diff --git a/howtos/preview-urls.md b/howtos/preview-urls.md
@@ -126,6 +126,31 @@ Need a sample app to try with preview URLs?  Check out the <a href="../../quick-
 
 ## Sharing a preview URL with people outside your team
 
-To collaborate with someone outside of your identity provider's organization, you must go to [Ambassador Cloud](https://app.getambassador.io/cloud/preview/), select the preview URL, and click **Make Publicly Accessible**.  Now anyone with the link will have access to the preview URL. When they visit the preview URL, they will see the intercepted service running on your laptop.
+To collaborate with someone outside of your identity provider's organization, you must go to [Ambassador Cloud](https://app.getambassador.io/cloud/), navigate to your service's intercepts, select the preview URL details, and click **Make Publicly Accessible**.  Now anyone with the link will have access to the preview URL. When they visit the preview URL, they will see the intercepted service running on your laptop.
 
-To disable sharing the preview URL publicly, click **Require Authentication** in the dashboard. Removing the intercept either from the dashboard or by running `telepresence leave <service-name>` also removes all access to the preview URL.
+To disable sharing the preview URL publicly, click **Require Authentication** in the dashboard. Removing the preview URL either from the dashboard or by running `telepresence preview remove <intercept-name>` also removes all access to the preview URL.
+
+## Change access restrictions
+
+To collaborate with someone outside of your identity provider's organization, you must make your preview URL publicly accessible.
+
+1. Go to [Ambassador Cloud](https://app.getambassador.io/cloud/)
+2. Navigate to the desired service Intercepts page
+3. Expand the preview URL details
+4. Click **Make Publicly Accessible**
+
+Now anyone with the link will have access to the preview URL. When they visit the preview URL, they will see the intercepted service running on a local environment.
+
+To disable sharing the preview URL publicly, click **Require Authentication** in the dashboard.
+
+## Remove a preview URL from an Intercept
+
+To delete a preview URL and remove all access to the intercepted service,
+
+1. Go to [Ambassador Cloud](https://app.getambassador.io/cloud/)
+2. Navigate to the desired service Intercepts page
+3. Expand the preview URL details
+4. Click **Remove Preview**
+
+Alternatively, a preview URL can also be removed by running
+`telepresence preview remove <intercept-name>`
diff --git a/howtos/request.md b/howtos/request.md
@@ -0,0 +1,12 @@
+import Alert from '@material-ui/lab/Alert';
+
+# Send requests to an intercepted service
+
+Ambassador Cloud can inform you about the required request parameters to reach an intercepted service.
+
+  1. Go to [Ambassador Cloud](https://app.getambassador.io/cloud/)
+  2. Navigate to the desired service Intercepts page
+  3. Click the **Query** button to open the pop-up menu.
+  4. Toggle between **CURL**, **Headers** and **Browse**.
+
+The pre-built queries and header information should help you get started to query the desired intercepted service and manage header propagation. 
diff --git a/reference/cluster-config.md b/reference/cluster-config.md
@@ -152,3 +152,21 @@ workload template's annotations:
      spec:
        containers:
 ```
+
+### Service Port Annotation
+
+A service port annotation can be added to the workload to make the Mutating Webhook select a specific port
+in the service. This is necessary when the service has multiple ports.
+
+```diff
+ spec:
+   template:
+     metadata:
+       labels:
+         service: your-service
+       annotations:
+         telepresence.getambassador.io/inject-traffic-agent: enabled
++        telepresence.getambassador.io/inject-service-port: https
+     spec:
+       containers:
+```
diff --git a/reference/config.md b/reference/config.md
@@ -35,7 +35,7 @@ These are the valid fields for the `timeouts` key:
 |`intercept`|Waiting for an intercept to become active|5 seconds|
 |`proxyDial`|Waiting for an outbound connection to be established|5 seconds|
 |`trafficManagerConnect`|Waiting for the Traffic Manager API to connect for port fowards|20 seconds|
-|`trafficManagerAPI`|Waiting for connection to the gPRC API after `trafficManagerConnect` is successful|5 seconds|
+|`trafficManagerAPI`|Waiting for connection to the gPRC API after `trafficManagerConnect` is successful|15 seconds|
 
 #### Log Levels
 Values for `logLevels` are one of the following strings: `trace`, `debug`, `info`, `warning`, `error`, `fatal` and `panic`.

diff --git a/reference/dns.md b/reference/dns.md
@@ -66,3 +66,5 @@ $ curl verylargejavaservice:8080
 Now curling that service by its short name works and will as long as the intercept is active.
 
 The DNS resolver will always be able to resolve services using `<service-name>.<namespace>` regardless of intercepts.
+
+See [Outbound connectivity](../routing/#dns-resolution) for details on DNS lookups.
diff --git a/reference/intercepts.md b/reference/intercepts.md
@@ -2,7 +2,7 @@ import Alert from '@material-ui/lab/Alert';
 
 # Intercepts
 
-## Intercept behvaior when logged into Ambassador Cloud
+## Intercept behavior when logged into Ambassador Cloud
 
 After logging into Ambassador Cloud (with `telepresence login`), Telepresence will default to `--preview-url=true`, which will use Ambassador Cloud to create a sharable preview URL for this intercept. (Creating an intercept without logging in will default to `--preview-url=false`).
 

diff --git a/reference/routing.md b/reference/routing.md
@@ -0,0 +1,46 @@
+# Connection Routing
+
+## Outbound
+
+### DNS resolution
+When requesting a connection to a host, the IP of that host must be determined. Telepresence provides DNS resolvers to help with this task. There are currently three types of resolvers but only one of them will be used on a workstation at any given time. Common for all of them is that they will propagate a selection of the host lookups to be performed in the cluster. The selection normally includes all names ending with `.cluster.local` or a currently mapped namespace but more entries can be added to the list using the `include-suffixes` option in the
+[local DNS configuration](../config/#dns) 
+
+#### Cluster side DNS lookups
+The cluster side host lookup will be performed by the traffic-manager unless the client has an active intercept, in which case, the agent performing that intercept will be responsible for doing it. If the client has multiple intercepts, then all of them will be asked to perform the lookup, and the response to the client will contain the unique sum of IPs that they produce. It's therefore important to never have multiple intercepts that span more than one namespace[<sup>[1](#namespacelimit)</sup>]. The reason for asking all of them is that the workstation currently impersonates multiple containers, and it is not possible to determine on behalf of what container the lookup request is made.
+
+#### macOS resolver
+This resolver hooks into the macOS DNS system by creating files under `/etc/resolver`. Those files correspond to some domain and contain the port number of the Telepresence resolver. Telepresence creates one such file for each of the currently mapped namespaces and `include-suffixes` option. The file `telepresence.local` contains a search path that is configured based on current intercepts so that single label names can be resolved correctly.
+
+#### Linux systemd-resolved resolver
+This resolver registers itself as part of telepresence's [VIF](../tun-device) using `systemd-resolved` and uses the DBus API to configure domains and routes that corresponds to the current set of intercepts and namespaces.
+
+#### Linux overriding resolver
+Linux systems that aren't configured with `systemd-resolved` will use this resolver. A Typical case is when running Telepresence [inside a docker container](../inside-container). During initialization, the resolver will first establish a _fallback_ connection to the IP passed as `--dns`, the one configured as `local-ip` in the [local DNS configuration](../config/#dns), or the primary `nameserver` registered in `/etc/resolv.conf`. It will then use iptables to actually override that IP so that requests to it instead end up in the overriding resolver, which unless it succeeds on its own, will use the _fallback_.
+
+### Routing
+
+#### Subnets
+The Telepresence `traffic-manager` service is responsible for discovering the cluster's Service subnet and all subnets used by the pods. In order to do this, it needs permission to create a dummy service[<sup>[2](#servicesubnet)</sup>] in its own namespace, and the ability to list, get, and watch nodes and pods. Some clusters will expose the pod subnets as `podCIDR` in the `Node` but some, like Amazon EKS, typically don't. Telepresence will then fall back to deriving the subnets from the IPs of all pods.
+
+The complete set of subnets that the [VIF](../tun-device) will be configured with is dynamic and may change during a connection's life cycle as new nodes arrive or disappear from the cluster. The set consists of what that the traffic-manager finds in the cluster, and the subnets configured using the [also-proxy](../config#alsoproxy) configuration option. Telepresence will remove subnets that are equal to, or completely covered by, other subnets.
+
+#### Connection origin
+A request to connect to an IP-address that belongs to one of the subnets of the [VIF](../tun-device) will cause a connection request to be made in the cluster. As with host name lookups, the request will originate from the traffic-manager unless the client has ongoing intercepts. If it does, one of the intercepted pods will be chosen, and the request will instead originate from that pod. This is a best-effort approach. Telepresence only knows that the request originated from the workstation. It cannot know that it is intended to originate from a specific pod when multiple intercepts are active.
+
+There are multiple reasons for doing this. One is that it is important that the request originates from the correct namespace. Example:
+
+```bash
+curl some-host
+```
+results in a http request with header `Host: some-host`. Now, if a service-mesh like Istio performs header based routing, then it will fail to find that host unless the request originates from the same namespace as the host resides in. Another reason is that the configuration of a service mesh can contain very strict rules. If the request then originates from the wrong pod, it will be denied. Only one intercept at a time can be used if there is a need to ensure that the chosen pod is exactly right.
+
+## Inbound
+
+The traffic-manager and traffic-agent are mutually responsible for setting up the necessary connection to the workstation when an intercept becomes active. In versions prior to 2.3.2, this would be accomplished by the traffic-manager creating a port dynamically that it would pass to the traffic-agent. The traffic-agent would then forward the intercepted connection to that port, and the traffic-manager would forward it to the workstation. This lead to problems when integrating with service meshes like Istio since those dynamic ports needed to be configured. It also imposed an undesired requirement to be able to use mTLS between the traffic-manager and traffic-agent.
+
+In 2.3.2, this changes, so that the traffic-agent instead creates a tunnel to the traffic-manager using the already existing gRPC API connection. The traffic-manager then forwards that using another tunnel to the workstation. This is completely invisible to other service meshes and is therefore much easier to configure.
+
+##### Footnotes:
+<p><a name="namespacelimit">1</a>: A future version of Telepresence will not allow concurrent intercepts that span multiple namespaces.</p>
+<p><a name="servicesubnet">2</a>: The error message from an attempt to create a service in a bad subnet contains the service subnet. The trick of creating a dummy service is currently the only way to get Kubernetes to expose that subnet.</p>
diff --git a/releaseNotes.yml b/releaseNotes.yml
@@ -30,6 +30,110 @@ docDescription: >-
 changelog: https://github.com/telepresenceio/telepresence/blob/$branch$/CHANGELOG.md
 
 items:
+  - version: 2.3.2
+    date: '2021-06-18'
+    notes:
+      # Headliners
+      - type: feature
+        title: Service Port Annotation
+        body: >-
+          The mutator webhook for injecting traffic-agents now
+          recognizes a
+          <code>telepresence.getambassador.io/inject-service-port</code>
+          annotation to specify which port to intercept; bringing the
+          functionality of the <code>--port</code> flag to users who
+          use the mutator webook in order to control Telepresence via
+          GitOps.
+        image: ./telepresence-2.3.2-svcport-annotation.png
+        docs: reference/cluster-config#service-port-annotation
+      - type: feature
+        title: Outbound Connections
+        body: >-
+          Outbound connections are now routed through the intercepted
+          Pods which means that the connections originate from that
+          Pod from the cluster's perspective.  This allows service
+          meshes to correctly identify the traffic.
+        docs: reference/routing/#outbound
+      - type: change
+        title: Inbound Connections
+        body: >-
+          Inbound connections from an intercepted agent are now
+          tunneled to the manager over the existing gRPC connection,
+          instead of establishing a new connection to the manager for
+          each inbound connection.  This avoids interference from
+          certain service mesh configurations.
+        docs: reference/routing/#inbound
+
+      # RBAC changes
+      - type: change
+        title: Traffic-manager needs new RBAC permissions
+        body: >-
+          The traffic-manager requires <a href="reference/rbac">RBAC
+          permissions</a> to list Nodes, Pods, and to create a dummy
+          Service in the manager's namespace.
+        docs: reference/routing/#subnets
+      - type: change
+        title: Reduced developer RBAC requirements
+        body: >-
+          The on-laptop client no longer requires <a
+          href="reference/rbac">RBAC permissions</a> to list the Nodes
+          in the cluster or to create Services, as that functionality
+          has been moved to the traffic-manager.
+
+      # Bugfixes
+      - type: bugfix
+        title: Able to detect subnets
+        body: >-
+          Telepresence will now detect the Pod CIDR ranges even if
+          they are not listed in the Nodes.
+        image: ./telepresence-2.3.2-subnets.png
+        docs: reference/routing/#subnets
+      - type: bugfix
+        title: Dynamic IP ranges
+        body: >-
+          The list of cluster subnets that the virtual network
+          interface will route is now configured dynamically and will
+          follow changes in the cluster.
+      - type: bugfix
+        title: No duplicate subnets
+        body: >-
+          Subnets fully covered by other subnets are now pruned
+          internally and thus never superfluously added to the
+          laptop's routing table.
+        docs: reference/routing/#subnets
+      - type: change # not a bugfix, but it only makes sense to mention after the above bugfixes
+        title: Change in default timeout
+        body: >-
+          The <code>trafficManagerAPI</code> timeout default has
+          changed from 5 seconds to 15 seconds, in order to facilitate
+          the extended time it takes for the traffic-manager to do its
+          initial discovery of cluster info as a result of the above
+          bugfixes.
+      - type: bugfix
+        title: Removal of DNS config files on macOS
+        body: >-
+          On macOS, files generated under
+          <code>/etc/resolver/</code> as the result of using
+          <code>include-suffixes</code> in the cluster config are now
+          properly removed on quit.
+        docs: reference/routing/#mac-os-resolver
+
+      - type: bugfix
+        title: Large file transfers
+        body: >-
+          Telepresence no longer erroneously terminates connections
+          early when sending a large HTTP response from an intercepted
+          service.
+      - type: bugfix
+        title: Race condition in shutdown
+        body: >-
+          When shutting down the user-daemon or root-daemon on the
+          laptop, <code>telepresence quit</code> and related commands
+          no longer return early before everything is fully shut down.
+          Now it can be counted on that by the time the command has
+          returned that all of the side-effects on the laptop have
+          been cleaned up.
+
   - version: 2.3.1
     date: '2021-06-14'
     notes:

diff --git a/versions.yml b/versions.yml
@@ -1,2 +1,3 @@
+version: 2.3.2
 docsVersion: "pre-release"
 branch: release/v2