Skip to content

Getting Started

Jump to bottom
Gary Vermeulen edited this page May 20, 2024 · 42 revisions

Index

Overview

The Layer7 Operator completely transforms the deployment, configuration and management of Layer7 API Security in Kubernetes. It helps customers unlock the many benefits of cloud native deployments and upgrades, including greater agility, availability and scalability; faster times to market; and reduced costs.

Kubernetes operators automate tasks traditionally performed by a human operator of an application. Kubernetes operators should have the same level of deployment, behavior and troubleshooting expertise as a human operator when automating those tasks.

Kubernetes operators are Kubernetes extensions created using the Operator SDK. They make use of custom resources to manage applications and their components adhering to the Kubernetes principle of control loops. A control loop constantly observes the desired state of resources, analyzes the actual state of those resources, and executes actions to synchronize the current state with the desired state as necessary.

The Layer7 Operator has expert knowledge of Layer7 products, and in the preview release, it makes use of gateway and repository custom resources to automate the deployment, configuration and management of Layer7 API Gateways using static or dynamic config-as-code in the form of Graphman bundles from different sources of truth including git repositories or artifact repositories (or optionally backed by a MySQL database).

Here are the highlights of the Layer7 Operator preview release:

  • Feature Parity with Gateway Helm Deployments

    • The Layer7 Operator supports the same deployment options for the Layer7 API Gateway as its Helm chart does, including support for initContainers but excluding sub-charts to deploy optional integrations.

  • Helm Deployment for the Layer7 Operator

    • The Layer7 Operator itself can be deployed using Helm. Then, the Layer7 Operator can deploy and manage gateways using Gateway custom resources instead of Helm.

  • Operator Hub Deployment for the Layer7 Operator

  • Graphman Integration

    • The Layer7 Operator itself uses Graphman to dynamically apply configuration to managed gateways.

  • Static and Dynamic Config as Code for Ephemeral Gateways

    • The Layer7 Operator automates the configuration of ephemeral container gateways by applying Graphman bundles statically on gateway startup or dynamically to running gateways without having to restart them.

  • Git and Artifact Repository Config Sources of Truth

    • The Layer7 Operator can monitor git and artifact repositories for changes to exploded or imploded Graphman bundle content, and pull and cache those changes to efficiently and automatically apply them to managed gateways.

  • External Secrets Management

    • The Layer7 Operator can synchronize external secrets via the Kubernetes External Secrets Operator with Layer7 API Gateway stored passwords.

Prerequisites

  • Kubernetes
    • This getting started guide assumes you already have a Kubernetes Cluster up and running. If you do not have a Kubernetes Cluster available you can try out the Layer7 Operator with Kind (Kubernetes in Docker). Check out this guide to get set up.
  • kubectl
  • cert-manager
    • Only required for the cluster deployment option, and optionally installed as part of the cluster deployment steps in this guide if not already available in the cluster
  • Layer7 API Gateway License (for version deployed)
  • Layer7 Policy Manager (for version deployed; and optional)

Compatibility Matrix

The Layer7 Operator graphman client is strongly typed, meaning changes to the Graphman Schema can cause breaking changes between versions of the Operator/Gateways.

Operator Version Gateway Versions
v1.0.6 11.1.00
v1.0.5 10.1.00_CR4, 11.0.00_CR2
v1.0.4 10.1.00_CR3, 11.0.00_CR1

Deploy the Layer7 Operator

The Layer7 Operator can be deployed using Helm, Operator Hub and Openshift though this getting started guide demonstrates deploying the Layer7 Operator using kubectl commands.

The following commands will deploy the Operator with all of it's required resources. Choose either a cluster deployment or a namespaced deployment, not both.

With a cluster deployment, the Operator will monitor all namespaces for Layer7 custom resources by default, though this can be configured. With a namespaced deployment, the Operator will only monitor the namespace it's deployed to for Layer7 customer resources. In most cases, a cluster deployment is recommended. Cluster deployments include validating and mutating webhooks that can automatically reject invalid configurations and handle Gateway and Repository Custom Resources across upgrades which may include API changes. Cluster deployments are also only deployed once and are therefore simpler to upgrade.

Again, in most cases, a cluster deployment is recommended. However, in some cases, a namespaced deployment may be appropriate. Those cases include when multiple people are following this getting started guide in a shared cluster at the same time. Namespaced deployments will help avoid collisions between those people.

Cluster Deployment

Install Cert Manager

The Cluster deployment requires cert-manager for the Operators Validating and Mutating WebhookConfigurations.

You can install cert-manager with kubectl. There are additional options here

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.3/cert-manager.yaml

Note: You should watch the status of the cert manager deployment, and wait until all resources are ready before proceeding to the next step. Depending on your environment, and how fast resources deploy, you may want to do something similar after other steps of this getting started guide.

Linux/MacOS 
watch kubectl get all -n cert-manager
Windows

For lack of a watch command equivalent, you may have to repeat the following command multiple times until all resources report ready:

kubectl get all -n cert-manager

Deploy the Layer7 Operator

kubectl apply -f https://github.com/CAAPIM/layer7-operator/releases/download/v1.0.6/cw-bundle.yaml

Watch for the Layer7 Operator pod to be ready:

Linux/MacOS 
watch kubectl get all -n layer7-operator-system
Windows

For lack of a watch command equivalent, you may have to repeat the following command multiple times until all resources report ready:

kubectl get all -n layer7-operator-system

Note: By default, for cluster deployments, the Layer7 Operator monitors for new or updated Layer7 custom resources (e.g. gateway and repository customer resources) in all namespaces. To have the Layer7 Operator only monitor specific namespaces, download cw-bundle.yaml, update it as follows, and apply it with kubectl.

Default:

env:
  - name: WATCH_NAMESPACE
    value: ""

Watch specific namespaces:

env:
  - name: WATCH_NAMESPACE
    value: "ns1,ns2,ns3"
Namespaced Deployment

If you don't already have a namespace to deploy to, then create one:

kubectl create namespace <namespace_name>

Set your kubectl context to the namespace that you want to deploy to:

kubectl config set-context --current --namespace=<namespace_name>

Deploy the Layer7 Operator

kubectl apply -f https://github.com/CAAPIM/layer7-operator/releases/download/v1.0.6/bundle.yaml

Watch for the Layer7 Operator pod to be ready:

Linux/MacOS 
watch kubectl get all
Windows

For lack of a watch command equivalent, you may have to repeat the following command multiple times until all resources report ready:

kubectl get all

View the Layer7 Operator logs

View the Layer7 Operator logs to see how it manages new and updated gateway and repository custom resources during this getting started guide.

Cluster Deployment

If you deployed the Operator to watch all, multiple or a specific Namespace (cw-bundle.yaml - Cluster Deployment) you can use the following to get the Operator logs

kubectl logs -f -l control-plane=controller-manager -c manager -n layer7-operator-system
Namespaced Deployment

If you deployed the Operator in Namespaced mode (or using bundle.yaml) you can use the following to get the Operator logs

kubectl logs -f -l control-plane=controller-manager -c manager

Create a Repository

The Repository Custom Resource allows you to define a Git or HTTP endpoint that contains Graphman bundles (not Restman bundles). The repository can contain multiple Graphman bundles generated by the graphman-client. These can be in imploded (a single Graphman JSON bundle) or exploded (directory structure) format.

There are 3 example repositories that we make use of in our examples.

All three repositories are in exploded format and make up a reference implementation where Gateway Configuration/Services are split into 3 parts. This and subsequent configuration examples are not required or enforced, they represent one potential way to make use of Repositories to configure Gateways.

Defining a Repository

Here is a simple repository from our examples without auth. Please checkout this for basic/token auth or this for ssh auth. Also checkout the Repository API Reference for more configuration options.

Save this to a file called myrepository.yaml:

apiVersion: security.brcmlabs.com/v1
kind: Repository
metadata:
  name: l7-gw-myapis
spec:
  enabled: true
  type: git
  endpoint: https://github.com/Gazza7205/l7GWMyAPIs
  branch: main

Create the repository:

kubectl apply -f myrepository.yaml

If you are still tailing the Layer7 Operator log file, you should see log events when it detects the new repository custom resource.

Inspect Repositories

The Repository Controller will attempt to reconcile the Git or HTTP reference for use on Layer7 Gateways.

Get repositories:

kubectl get repositories

output

NAME           AGE
l7-gw-myapis   7s

Get the l7-gw-myapis repository in more detail

kubectl get repository l7-gw-myapis  -oyaml

output - pay special attention to status where the current commit is synchronized. If the git repository is updated, it will reflect there.

apiVersion: security.brcmlabs.com/v1
kind: Repository
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"security.brcmlabs.com/v1","kind":"Repository","metadata":{"annotations":{},"name":"l7-gw-myapis","namespace":"default"},"spec":{"auth":{},"branch":"main","enabled":true,"endpoint":"https://github.com/Gazza7205/l7GWMyAPIs","name":"l7-gw-myapis"}}
  creationTimestamp: "2023-11-13T12:12:37Z"
  generation: 1
  name: l7-gw-myapis
  namespace: default
  resourceVersion: "1983836"
  uid: 4f4a6380-27b7-4eb8-a599-69416375e2fb
spec:
  auth: {}
  branch: main
  enabled: true
  endpoint: https://github.com/Gazza7205/l7GWMyAPIs
  type: git
status:
  commit: 8fc74669689abe781645dac214ebf26eb7480c78
  name: l7-gw-myapis
  ready: true
  storageSecretName: l7-gw-myapis-repository-main
  updated: 2023-11-13 12:12:38.401873038 +0000 UTC m=+693.632725635

Deploy a Layer7 API Gateway

The Gateway Custom Resource allows you to define a Layer7 API Gateway with a variety of configurations. This will be a basic introduction. These examples cover more advanced scenarios (including basic, advanced and Open Telemetry examples using Elastic Stack or Prometheus). Also checkout the Gateway API Reference for more configuration options.

Create a Kubernetes Secret with your Gateway license:

NOTE: Replace the example license file path below with your path to a valid XML formatted license file for your gateway version.

Linux/MacOS 
kubectl create secret generic gateway-license --from-file=license.xml=./path/to/license/file.xml
Windows 
kubectl create secret generic gateway-license --from-file=license.xml=.\path\to\license\file.xml

The following Gateway definition is very basic, with a single repository reference. Repository references can also contain encrypted values, checkout the API reference for more details.

Save this to a file called mygateway.yaml:

apiVersion: security.brcmlabs.com/v1
kind: Gateway
metadata:
  name: ssg
spec:
  version: "11.1.00"
  license:
    accept: true
    secretName: gateway-license
  app:
    replicas: 1
    image: docker.io/caapim/gateway:11.1.00
    management:
      graphman:
        enabled: true
      username: admin
      password: 7layer
      cluster:
        password: 7layer
        hostname: gateway.brcmlabs.com
    service:
      type: LoadBalancer
      ports:
      - name: https
        port: 8443
        targetPort: 8443
        protocol: TCP
      - name: management
        port: 9443
        targetPort: 9443
        protocol: TCP
    repositoryReferences:
      - name: l7-gw-myapis
        enabled: true
        type: dynamic

Create the gateway:

kubectl apply -f mygateway.yaml

If you are still tailing the Layer7 Operator log file, you should see log events when it detects the new gateway custom resource.

Inspect Gateways

You can inspect the Gateway Custom Resource in the same way you did repositories again tracking status to see which configurations have been applied.

kubectl get gateway ssg -oyaml

Test your Gateway Deployment

Now that you've deployed a Gateway, you can connect with Policy Manager to view its configuration or with an API client to send API requests.

First, get the Gateway service:

kubectl get svc

Our example uses a service type of LoadBalancer. If your Kubernetes environment has a LoadBalancer provisioner and is able to provision an EXTERNAL-IP for the Gateway service, the result may look like this and you can use the EXTERNAL-IP address provided:

NAME  TYPE           CLUSTER-IP     EXTERNAL-IP         PORT(S)                         AGE
ssg   LoadBalancer   10.68.4.161    ***34.89.84.69***   8443:31747/TCP,9443:30778/TCP   41m

If your Kubernetes environment does not have a LoadBalancer provisioner or your environment is restricted from allocating external IP addresses, your Gateway service will look like this with the EXTERNAL-IP address in the state and you can use port forwarding instead:

NAME  TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                         AGE
ssg   LoadBalancer   10.68.4.126   <PENDING>       8443:31384/TCP,9443:31359/TCP   7m39s

Service Annotations are used by Kubernetes Vendors to create Internal LoadBalancers where these restrictions apply. Below are two examples for GKE and AKS, each vendor maintains their own supported annotations, it's always best to check their documentation to find the latest configuration recommendations/options.

app:
  service:
    annotations: {}
    #  cloud.google.com/load-balancer-type: "Internal"                  <<== Google GKE
    #  service.beta.kubernetes.io/azure-load-balancer-internal: "true"  <<== Azure AKS
    type: LoadBalancer
    ports:
    - name: https
      port: 8443
      targetPort: 8443
      protocol: TCP
    - name: management
      port: 9443
      targetPort: 9443
      protocol: TCP

The following sections will assume that you do not have an EXTERNAL-IP available, and will use port forwarding to connect with Policy Manager and to an API.

Connect with Policy Manager

Though we have deployed an ephemeral Gateway whose configuration we're managing with Gateway and Repository custom resources, and you would not normally want or need to connect to such Gateways using Policy Manager, our example has defined a service port for Management traffic on 9443 that can be connected to with Policy Manager.

  • Open a new terminal
    • kubectl port-forward svc/ssg 9443
  • Connect through Policy Manager
    • username: admin
    • password: 7layer
    • gateway: localhost:9443

Call an API

Our example has also defined a service port for Gateway traffic on 8443 that can be used to call APIs on the Gateway.

In Policy Manager you will see a folder called myApis that contains 4 APIS. Calling Rest Api 1 and Rest Api 2 will fail because they depend on policy fragments that are part of the framework repository. These can be added by following the examples here.

Rest Api 3 has no dependencies and should return a valid response. To try it out, port forward to 8443:

kubectl port-forward svc/ssg 8443

And use curl to call the API:

curl -k https://localhost:8443/api3

You should get a response like this:

{
    "message": "Thanks for getting started at 2024-02-11T20:45:04.304Z!"
}

Uninstall

Run the following commands to remove all of the components that you installed during this tutorial.

  • Remove Gateway
kubectl delete -f mygateway.yaml
  • Remove Repository
kubectl delete -f myrepository.yaml
  • Remove Gateway License
kubectl delete secret gateway-license
  • Remove Operator
Namespaced Deployment

If you created a namespace that you no longer need:

kubectl delete namespace <namespace_name>

If you are sharing your cluster with other people going through this getting started guide at the same time, then stop here. The following command to uninstall the Layer7 Operator will also remove the cluster wide custom resource definitions being used by those people. Those can be removed by having the last person to complete this getting started guide execute the following command.

kubectl delete -f https://github.com/CAAPIM/layer7-operator/releases/download/v1.0.6/bundle.yaml
Cluster Deployment 
kubectl delete -f https://github.com/CAAPIM/layer7-operator/releases/download/v1.0.6/cw-bundle.yaml

If you deployed cert-manager

kubectl delete -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.3/cert-manager.yaml

Next Steps

Share Your Experience

Now that you have gone through the getting started exercise yourself, share your experience with others by sharing a link to this getting started recording!

Have Fun!

Learn more about Graphman and the graphman-client. Then try using both to export Layer7 API Gateway configuration to a Graphman bundle, explode the Graphman bundle to a folder, add the contents of the folder to a new GitHub repository that you create, create a new repository custom resource and update your gateway custom resource to reference the repository. Then make changes to the gateway configuration in the GitHub repository to see those changes dynamically applied to your gateway by the Layer7 Operator.

Back to Home

Contents

Clone this wiki locally