openshift · JStickler · Oct 30, 2020 · Oct 9, 2020
diff --git a/modules/ossm-cr-threescale-1x.adoc b/modules/ossm-cr-threescale-1x.adoc
@@ -0,0 +1,104 @@
+// Module included in the following assemblies:
+//
+// * service_mesh/v1x/customizing-installation-ossm.adoc
+
+[id="ossm-cr-threescale-1x_{context}"]
+
+= 3scale configuration
+
+Here is an example that illustrates the 3scale Istio Adapter parameters for the {ProductName} custom resource and a description of the available parameters with appropriate values.
+
+.Example 3scale parameters
+[source,yaml]
+----
+threeScale:
+  enabled: false
+  PARAM_THREESCALE_LISTEN_ADDR: 3333
+  PARAM_THREESCALE_LOG_LEVEL: info
+  PARAM_THREESCALE_LOG_JSON: true
+  PARAM_THREESCALE_LOG_GRPC: false
+  PARAM_THREESCALE_REPORT_METRICS: true
+  PARAM_THREESCALE_METRICS_PORT: 8080
+  PARAM_THREESCALE_CACHE_TTL_SECONDS: 300
+  PARAM_THREESCALE_CACHE_REFRESH_SECONDS: 180
+  PARAM_THREESCALE_CACHE_ENTRIES_MAX: 1000
+  PARAM_THREESCALE_CACHE_REFRESH_RETRIES: 1
+  PARAM_THREESCALE_ALLOW_INSECURE_CONN: false
+  PARAM_THREESCALE_CLIENT_TIMEOUT_SECONDS: 10
+  PARAM_THREESCALE_GRPC_CONN_MAX_SECONDS: 60
+----
+
+.3scale parameters
+|===
+|Parameter |Description |Values |Default value
+
+|`enabled`
+|Whether to use the 3scale adapter
+|`true`/`false`
+|`false`
+
+|`PARAM_THREESCALE_LISTEN_ADDR`
+|Sets the listen address for the gRPC server
+|Valid port number
+|`3333`
+
+|`PARAM_THREESCALE_LOG_LEVEL`
+|Sets the minimum log output level.
+|`debug`, `info`, `warn`, `error`, or `none`
+|`info`
+
+|`PARAM_THREESCALE_LOG_JSON`
+|Controls whether the log is formatted as JSON
+|`true`/`false`
+|`true`
+
+|`PARAM_THREESCALE_LOG_GRPC`
+|Controls whether the log contains gRPC info
+|`true`/`false`
+|`true`
+
+|`PARAM_THREESCALE_REPORT_METRICS`
+|Controls whether 3scale system and backend metrics are collected and reported to Prometheus
+|`true`/`false`
+|`true`
+
+|`PARAM_THREESCALE_METRICS_PORT`
+|Sets the port that the 3scale `/metrics` endpoint can be scrapped from
+|Valid port number
+|`8080`
+
+|`PARAM_THREESCALE_CACHE_TTL_SECONDS`
+|Time period, in seconds, to wait before purging expired items from the cache
+|Time period in seconds
+|`300`
+
+|`PARAM_THREESCALE_CACHE_REFRESH_SECONDS`
+|Time period before expiry when cache elements are attempted to be refreshed
+|Time period in seconds
+|`180`
+
+|`PARAM_THREESCALE_CACHE_ENTRIES_MAX`
+|Max number of items that can be stored in the cache at any time. Set to `0` to disable caching
+|Valid number
+|`1000`
+
+|`PARAM_THREESCALE_CACHE_REFRESH_RETRIES`
+|The number of times unreachable hosts are retried during a cache update loop
+|Valid number
+|`1`
+
+|`PARAM_THREESCALE_ALLOW_INSECURE_CONN`
+|Allow to skip certificate verification when calling `3scale` APIs. Enabling this is not recommended.
+|`true`/`false`
+|`false`
+
+|`PARAM_THREESCALE_CLIENT_TIMEOUT_SECONDS`
+|Sets the number of seconds to wait before terminating requests to 3scale System and Backend
+|Time period in seconds
+|`10`
+
+|`PARAM_THREESCALE_GRPC_CONN_MAX_SECONDS`
+|Sets the maximum amount of seconds (+/-10% jitter) a connection may exist before it is closed
+|Time period in seconds
+|60
+|===
diff --git a/modules/ossm-cr-threescale.adoc b/modules/ossm-cr-threescale.adoc
@@ -27,6 +27,9 @@ threeScale:
   PARAM_THREESCALE_ALLOW_INSECURE_CONN: false
   PARAM_THREESCALE_CLIENT_TIMEOUT_SECONDS: 10
   PARAM_THREESCALE_GRPC_CONN_MAX_SECONDS: 60
+  PARAM_USE_CACHED_BACKEND: false
+  PARAM_BACKEND_CACHE_FLUSH_INTERVAL_SECONDS: 15
+  PARAM_BACKEND_CACHE_POLICY_FAIL_CLOSED: true
 ----
 
 .3scale parameters
@@ -102,4 +105,20 @@ threeScale:
 |Sets the maximum amount of seconds (+/-10% jitter) a connection may exist before it is closed
 |Time period in seconds
 |60
+
+
+|`PARAM_USE_CACHE_BACKEND`
+|If true, attempt to create an in-memory apisonator cache for authorization requests
+|`true`/`false`
+|`false`
+
+|`PARAM_BACKEND_CACHE_FLUSH_INTERVAL_SECONDS`
+|If the backend cache is enabled, this sets the interval in seconds for flushing the cache against 3scale
+|Time period in seconds
+|15
+
+|`PARAM_BACKEND_CACHE_POLICY_FAIL_CLOSED`
+|Whenever the backend cache cannot retrieve authorization data, whether to deny (closed) or allow (open) requests
+|`true`/`false`
+|`true`
 |===
diff --git a/modules/ossm-threescale-authentication.adoc b/modules/ossm-threescale-authentication.adoc
@@ -107,29 +107,28 @@ You can manipulate this object using the methods described previously. In the ex
           service: destination.labels["service-mesh.3scale.net/service-id"] | ""
 ----
 
-For this integration to work correctly, OIDC must still be done in 3scale for the client to be created in the identity provider (IdP). You should create link:https://istio.io/docs/ops/security/end-user-auth/[end-user authentication] for the service you want to protect in the same namespace as that service. The JWT is passed in the `Authorization` header of the request.
+For this integration to work correctly, OIDC must still be done in 3scale for the client to be created in the identity provider (IdP). You should create a link:https://istio.io/latest/docs/tasks/security/authorization/authz-jwt/[Request authorization] for the service you want to protect in the same namespace as that service. The JWT is passed in the `Authorization` header of the request.
 
-In the sample `Policy` defined below, replace `issuer` and `jwksUri` as appropriate.
+In the sample `RequestAuthentication` defined below, replace `issuer`, `jwksUri`, and `selector` as appropriate.
 
 .OpenID Policy example
 
 [source,yaml]
 ----
-  apiVersion: authentication.istio.io/v1alpha1
-  kind: Policy
-  metadata:
-    name: jwt-example
-    namespace: bookinfo
-  spec:
-    origins:
-      - jwt:
-          issuer: >-
-            http://keycloak-keycloak.34.242.107.254.nip.io/auth/realms/3scale-keycloak
-          jwksUri: >-
-  http://keycloak-keycloak.34.242.107.254.nip.io/auth/realms/3scale-keycloak/protocol/openid-connect/certs
-    principalBinding: USE_ORIGIN
-    targets:
-      - name: productpage
+apiVersion: security.istio.io/v1beta1
+kind: RequestAuthentication
+metadata:
+  name: jwt-example
+  namespace: bookinfo
+spec:
+  selector:
+    matchLabels:
+      app: productpage
+  jwtRules:
+  - issuer: >-
+      http://keycloak-keycloak.34.242.107.254.nip.io/auth/realms/3scale-keycloak
+    jwksUri: >-
+      http://keycloak-keycloak.34.242.107.254.nip.io/auth/realms/3scale-keycloak/protocol/openid-connect/certs
 ----
 
 [id="ossm-threescale-hybrid-authentication_{context}"]

diff --git a/modules/ossm-threescale-integrate-1x.adoc b/modules/ossm-threescale-integrate-1x.adoc
@@ -0,0 +1,66 @@
+// Module included in the following assemblies:
+//
+// * service_mesh/v1x/threescale_adapter/threescale-adapter.adoc
+// * service_mesh/v2x/threescale_adapter/threescale-adapter.adoc
+
+[id="ossm-threescale-integrate-1x_{context}"]
+= Integrate the 3scale adapter with {ProductName}
+
+You can use these examples to configure requests to your services using the 3scale Istio Adapter.
+
+
+.Prerequisites:
+
+* {ProductName} version 1.x
+* A working 3scale account (link:https://www.3scale.net/signup/[SaaS] or link:https://access.redhat.com/documentation/en-us/red_hat_3scale_api_management/2.5/html/installing_3scale/onpremises-installation[3scale 2.5 On-Premises])
+* Enabling backend cache requires 3scale 2.9 or greater
+* {ProductName} prerequisites
+
+[NOTE]
+====
+To configure the 3scale Istio Adapter, refer to {ProductName} custom resources for instructions on adding adapter parameters to the custom resource file.
+====
+
+
+[NOTE]
+====
+Pay particular attention to the `kind: handler` resource. You must update this with your 3scale credentials and the service ID of the API you want to manage.
+====
+
+. Modify the handler configuration with your 3scale configuration.
++
+.Handler configuration example
+[source,yaml]
+----
+  apiVersion: "config.istio.io/v1alpha2"
+  kind: handler
+  metadata:
+   name: threescale
+  spec:
+   adapter: threescale
+   params:
+     service_id: "<SERVICE_ID>"
+     system_url: "https://<organization>-admin.3scale.net/"
+     access_token: "<ACCESS_TOKEN>"
+   connection:
+     address: "threescale-istio-adapter:3333"
+----
+
+Optionally, you can provide a `backend_url` field within the _params_ section to override the URL provided by the 3scale configuration. This may be useful if the adapter runs on the same cluster as the 3scale on-premise instance, and you wish to leverage the internal cluster DNS.
+
+. Modify the rule configuration with your 3scale configuration to dispatch the rule to the threescale handler.
++
+.Rule configuration example
+[source,yaml]
+----
+  apiVersion: "config.istio.io/v1alpha2"
+  kind: rule
+  metadata:
+    name: threescale
+  spec:
+    match: destination.labels["service-mesh.3scale.net"] == "true"
+    actions:
+      - handler: threescale.handler
+        instances:
+          - threescale-authorization.instance
+----
diff --git a/modules/ossm-threescale-integrate.adoc b/modules/ossm-threescale-integrate.adoc
@@ -11,10 +11,15 @@ You can use these examples to configure requests to your services using the 3sca
 
 .Prerequisites:
 
-* {ProductName} version 1.x
-* A working 3scale account (link:https://www.3scale.net/signup/[SaaS] or link:https://access.redhat.com/documentation/en-us/red_hat_3scale_api_management/2.5/html/installing_3scale/onpremises-installation[3scale 2.5 On-Premises])
+* {ProductName} version 2.x
+* A working 3scale account (link:https://www.3scale.net/signup/[SaaS] or link:https://access.redhat.com/documentation/en-us/red_hat_3scale_api_management/2.9/html/installing_3scale/install-threescale-on-openshift-guide[3scale 2.9 On-Premises])
+* Enabling backend cache requires 3scale 2.9 or greater
 * {ProductName} prerequisites
 * Ensure Mixer policy enforcement is enabled. Update Mixer policy enforcement section provides instructions to check the current Mixer policy enforcement status and enable policy enforcement.
+* Mixer policy and telemetry must be enabled if you are using a mixer plug-in.
+** You will need to properly configure the link:https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html/service_mesh/service-mesh-installation#ossm-control-plane-deploy-2x_installing-ossm-v2x[Service Mesh Control Plane (SMCP)] when upgrading.
+
+IMPORTANT: When you want to enable 3scale backend cache with the 3scale Istio adapter, you must also enable mixer policy and mixer telemetry. See xref:../../service_mesh/v2x/installing-ossm.adoc#ossm-control-plane-deploy_installing-ossm[Deploying the Red Hat OpenShift Service Mesh control plane].
 
 [NOTE]
 ====

diff --git a/modules/ossm-threescale-metrics-1x.adoc b/modules/ossm-threescale-metrics-1x.adoc
@@ -0,0 +1,8 @@
+// Module included in the following assemblies:
+//
+// * service_mesh/v1x/threescale_adapter/threescale-adapter.adoc
+// * service_mesh/v2x/threescale_adapter/threescale-adapter.adoc
+
+[id="ossm-threescale-metrics-1x_{context}"]
+= 3scale Adapter metrics
+The adapter, by default reports various Prometheus metrics that are exposed on port `8080` at the `/metrics` endpoint. These metrics provide insight into how the interactions between the adapter and 3scale are performing. The service is labeled to be automatically discovered and scraped by Prometheus.
diff --git a/modules/ossm-threescale-metrics.adoc b/modules/ossm-threescale-metrics.adoc
@@ -6,3 +6,28 @@
 [id="ossm-threescale-metrics_{context}"]
 = 3scale Adapter metrics
 The adapter, by default reports various Prometheus metrics that are exposed on port `8080` at the `/metrics` endpoint. These metrics provide insight into how the interactions between the adapter and 3scale are performing. The service is labeled to be automatically discovered and scraped by Prometheus.
+
+NOTE: There are incompatible changes in the 3scale Istio Adapter metrics since the previous releases in Service Mesh 1.x.
+
+In Prometheus, metrics have been renamed with one addition for the backend cache, so that the following metrics exist as of Service Mesh 2.0:
+
+.Prometheus metrics
+|===
+|Metric |Type |Description
+
+|`threescale_latency`
+|Histogram
+|Request latency between adapter and 3scale.
+
+|`threescale_http_total`
+|Counter
+|HTTP Status response codes for requests to 3scale backend.
+
+|`threescale_system_cache_hits`
+|Counter
+|Total number of requests to the 3scale system fetched from the configuration cache.
+
+|`threescale_backend_cache_hits`
+|Counter
+|Total number of requests to 3scale backend fetched from the backend cache.
+|===
diff --git a/modules/threescale-backend-cache.adoc b/modules/threescale-backend-cache.adoc
@@ -0,0 +1,45 @@
+// Module included in the following assemblies:
+//
+// * service_mesh/v1x/threescale_adapter/threescale-adapter.adoc
+// * service_mesh/v2x/threescale_adapter/threescale-adapter.adoc
+
+[id="threescale-backend-cache_{context}"]
+= 3scale backend cache
+
+The 3scale backend cache provides an authorization and reporting cache for clients of the 3scale Service Management API. This cache is embedded in the adapter to enable lower latencies in responses in certain situations assuming the administrator is willing to accept the trade-offs.
+
+NOTE: 3scale backend cache is disabled by default. 3scale backend cache functionality trades inaccuracy in rate limiting and potential loss of hits since the last flush was performed for low latency and higher consumption of resources in the processor and memory.
+
+== Advantages of enabling backend cache
+
+The following are advantages to enabling the backend cache:
+
+* Enable the backend cache when you find latencies are high while accessing services managed by the 3scale Istio Adapter.
+* Enabling the backend cache will stop the adapter from continually checking with the 3scale API manager for request authorizations, which will lower the latency.
+** This creates an in-memory cache of 3scale authorizations for the 3scale Istio Adapter to store and reuse before attempting to contact the 3scale API manager for authorizations. Authorizations will then take much less time to be granted or denied.
+* Backend caching is useful in cases when you are hosting the 3scale API manager in another geographical location from the service mesh running the 3scale Istio Adapter.
+** This is generally the case with the 3scale Hosted (SaaS) platform, but also if a user hosts their 3scale API manager in another cluster located in a different geographical location, in a different availability zone, or in any case where the network overhead to reach the 3scale API manager is noticeable.
+
+
+== Trade-offs for having lower latencies
+
+The following are trade-offs for having lower latencies:
+
+* Each 3scale adapter's authorization state updates every time a flush happens.
+** This means two or more instances of the adapter will introduce more inaccuracy between flushing periods.
+** There is a greater chance of too many requests being granted that exceed limits and introduce erratic behavior, which leads to some requests going through and some not, depending on which adapter processes each request.
+* An adapter cache that cannot flush its data and update its authorization information risks shut down or crashing without reporting its information to the API manager.
+* A fail open or fail closed policy will be applied when an adapter cache cannot determine whether a request must be granted or denied, possibly due to network connectivity issues in contacting the API manager.
+* When cache misses occur, typically right after booting the adapter or after a long period of no connectivity, latencies will grow in order to query the API manager.
+* An adapter cache must do much more work on computing authorizations than it would without an enabled cache, which will tax processor resources.
+* Memory requirements will grow proportionally to the combination of the amount of limits, applications, and services managed by the cache.
+
+== Backend cache configuration settings
+
+The following points explain the backend cache configuration settings:
+
+* Find the settings to configure the backend cache in the xref:../../service_mesh/v2x/threescale-adapter.adoc#ossm-cr-threescale_threescale-adapter[Example 3scale configuration options].
+* The last 3 settings control enabling of backend cache:
+** `PARAM_USE_CACHE_BACKEND` - set to true to enable backend cache.
+** `PARAM_BACKEND_CACHE_FLUSH_INTERVAL_SECONDS` - sets time in seconds between consecutive attempts to flush cache data to the API manager.
+** `PARAM_BACKEND_CACHE_POLICY_FAIL_CLOSED` - set whether or not to allow/open or deny/close requests to the services when there is not enough cached data and the 3scale API manager cannot be reached.
diff --git a/modules/threescale-istio-adapter-apicast.adoc b/modules/threescale-istio-adapter-apicast.adoc
@@ -0,0 +1,13 @@
+// Module included in the following assemblies:
+//
+// * service_mesh/v1x/threescale_adapter/threescale-adapter.adoc
+// * service_mesh/v2x/threescale_adapter/threescale-adapter.adoc
+
+[id="threescale-istio-adapter-apicast_{context}"]
+= 3scale Istio Adapter APIcast emulation
+
+The 3scale Istio Adapter performs as APIcast would when the following conditions occur:
+
+* When a request cannot match any mapping rule defined, the returned HTTP code is 404 Not Found. This was previously 403 Forbidden.
+* When a request is denied because it goes over limits, the returned HTTP code is 429 Too Many Requests. This was previously 403 Forbidden.
+* When generating default templates via the CLI, it will use underscores rather than dashes for the headers, for example: `user_key` rather than `user-key`.
diff --git a/service_mesh/v1x/customizing-installation-ossm.adoc b/service_mesh/v1x/customizing-installation-ossm.adoc
@@ -36,7 +36,7 @@ include::modules/ossm-jaeger-config-elasticsearch.adoc[leveloffset=+2]
 
 For more information about configuring Elasticsearch with {product-title}, see  xref:../../logging/config/cluster-logging-log-store.adoc[Configuring the log store].
 
-include::modules/ossm-cr-threescale.adoc[leveloffset=+1]
+include::modules/ossm-cr-threescale-1x.adoc[leveloffset=+1]
 
 
 == Next steps

diff --git a/service_mesh/v1x/threescale-adapter.adoc b/service_mesh/v1x/threescale-adapter.adoc
@@ -8,7 +8,7 @@ The 3scale Istio Adapter is an optional adapter that allows you to label a servi
 It is not required for {ProductName}.
 
 
-include::modules/ossm-threescale-integrate.adoc[leveloffset=+1]
+include::modules/ossm-threescale-integrate-1x.adoc[leveloffset=+1]
 
 include::modules/ossm-threescale-cr.adoc[leveloffset=+2]
 
@@ -24,4 +24,4 @@ include::modules/ossm-threescale-caching.adoc[leveloffset=+1]
 
 include::modules/ossm-threescale-authentication.adoc[leveloffset=+1]
 
-include::modules/ossm-threescale-metrics.adoc[leveloffset=+1]
+include::modules/ossm-threescale-metrics-1x.adoc[leveloffset=+1]
diff --git a/service_mesh/v2x/threescale-adapter.adoc b/service_mesh/v2x/threescale-adapter.adoc
@@ -25,3 +25,7 @@ include::modules/ossm-threescale-caching.adoc[leveloffset=+1]
 include::modules/ossm-threescale-authentication.adoc[leveloffset=+1]
 
 include::modules/ossm-threescale-metrics.adoc[leveloffset=+1]
+
+include::modules/threescale-backend-cache.adoc[leveloffset=+1]
+
+include::modules/threescale-istio-adapter-apicast.adoc[leveloffset=+1]