Kiali project, observability for the Istio service mesh
Clone or download
jshaughn KIALI-1670 Support option groupBy=app to group nodes by app label (#659)
* KIALI-1670 Support option groupBy=app
- allows grouping (via compound nodes) by app label value
also:
- add business service GetServiceDefinition() to fetch basic service info. This
  is to satisfy a need in the graph appender that only needs limited info (labels).

* Lponce PR Feedback
Latest commit 886d394 Nov 20, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Update bug_report.md Jul 23, 2018
business KIALI-1670 Support option groupBy=app to group nodes by app label (#659) Nov 20, 2018
config gofmt 1.11 has changed Nov 13, 2018
deploy go back to using major/minor - this should be OK (#663) Nov 15, 2018
graph KIALI-1670 Support option groupBy=app to group nodes by app label (#659) Nov 20, 2018
hack [hack] provide an option to wait for istio components to deploy befor… Nov 15, 2018
handlers KIALI-1670 Support option groupBy=app to group nodes by app label (#659) Nov 20, 2018
kubernetes KIALI-1915 Refactor IstioObject types Nov 13, 2018
log initial skeleton Feb 3, 2018
models Merge pull request #647 from israel-hdez/kiali-1873 Nov 15, 2018
prometheus KIALI-1856 no need for this metric - it is redundant. we get the same… Nov 8, 2018
routing KIALI-1542 Improve namespaces graph endpoint (#622) Nov 6, 2018
server Update format Oct 30, 2018
status KIALI-1587 Jaeger link tries to open popup (#573) Oct 18, 2018
tests Merge pull request #657 from lucasponce/refactor-istio-rules Nov 14, 2018
util KIALI-1714 Limit the lower bound of Prometheus queries (#560) Oct 22, 2018
vendor Add kube-openapi dependency Oct 29, 2018
.gitignore Exclude .kiali-e2e from repo Oct 25, 2018
.travis.yml Control that the swagger documentation is correct (#574) Oct 23, 2018
CODE_OF_CONDUCT.md Add contacts. Jul 20, 2018
CONTRIBUTING.md Further additions Oct 8, 2018
DEMOS.adoc Add a list of mesh examples (#333) Jul 6, 2018
LICENSE Initial commit Feb 3, 2018
Makefile KIALI-1930 Splitting Kubernetes Files (#665) Nov 14, 2018
PULL_REQUEST_TEMPLATE.md Create PULL_REQUEST_TEMPLATE.md (#366) Jul 31, 2018
README.adoc KIALI-1843 Enable/disable k8s cache from config (#610) Oct 31, 2018
RELEASING.adoc update release docs to use one patch branch (#662) Nov 13, 2018
STYLE_GUIDE.adoc KIALI-454 Contributing section (readme) - more in style guide Jun 21, 2018
config.yaml Change default path for console (#515) Sep 24, 2018
doc.go KIALI-1670 Support option groupBy=app to group nodes by app label (#659) Nov 20, 2018
glide.lock Add kube-openapi dependency Oct 29, 2018
glide.yaml Add kube-openapi dependency Oct 29, 2018
kiali.go KIALI-1820 prometheus scrape endpoint (#598) Oct 29, 2018
swagger.json KIALI-1670 Support option groupBy=app to group nodes by app label (#659) Nov 20, 2018

README.adoc

Kiali Tweet about Kiali

Build Status Apache 2.0 license

Introduction

Kiali provides answers to the questions: What microservices are part of my Istio service mesh and how are they connected?

Kiali Graph

Table of contents

Description

A Microservice Architecture breaks up the monolith into many smaller pieces that are composed together. Patterns to secure the communication between services like fault tolerance (via timeout, retry, circuit breaking, etc.) have come up as well as distributed tracing to be able to see where calls are going.

A service mesh can now provide these services on a platform level and frees the application writers from those tasks. Routing decisions are done at the mesh level.

Kiali works with Istio to visualise the service mesh topology, features like circuit breakers or request rates.

Kiali also includes Jaeger Tracing to provide distributed tracing out of the box.

Getting Started

If you are not a developer but just want to run something to see Kiali, then go to https://www.kiali.io for some instructions.

You could also try to use our hack script to quickly install OpenShift, Istio, and Kiali like this:

wget https://raw.githubusercontent.com/kiali/kiali/master/hack/cluster-openshift.sh
sh cluster-openshift.sh --kiali-enabled true up

If you do not already have an Istio-enabled application to test with, you can install one using the Bookinfo Demo install script provided here as a convenience.

There are other demo applications to choose from as well: see some examples.

Docker Image

Kiali is published as a docker image on Docker hub at kiali/kiali

See the LICENSE file.

Building

Note
These build instructions assume you have the following installed on your system: (1) Go Programming Language which must be at least version 1.8.3, (2) git, (3) Docker, and (4) make. To run Kiali on OpenShift after you build it, it is assumed you have a running OpenShift environment available to you. If you do not, you can find a set of instructions on how to set up OpenShift below. To run Kiali on Kubernetes after you built it, it is assumed you have a running Kubernetes environment available to you.

To build Kiali:

  • Clone this repository inside a GOPATH. These instructions will use the example GOPATH of "/source/kiali/kiali" but you can use whatever you want. Just change the first line of the below instructions to use your GOPATH.

export GOPATH=/source/kiali/kiali
mkdir -p $GOPATH
cd $GOPATH
mkdir -p src/github.com/kiali
cd src/github.com/kiali
git clone git@github.com:kiali/kiali
export PATH=${PATH}:${GOPATH}/bin
  • Install Glide - the Go dependency management tool that Kiali uses to build itself

cd ${GOPATH}/src/github.com/kiali/kiali
make dep-install
  • Tell the dependency manager tool to update the Kiali dependencies

    Note
    You should only run this command if you add, remove, or modify a dependency. If you are simply git cloning and building from source, you should skip this step.
cd ${GOPATH}/src/github.com/kiali/kiali
make dep-update
  • Build Kiali

cd ${GOPATH}/src/github.com/kiali/kiali
make build
  • At this point you can run the Kiali tests

cd ${GOPATH}/src/github.com/kiali/kiali
make test

Running

Note
If you want to quickly get up and running to play with Kiali and do not wish to git clone the repository, or build anything, see https://kiali.org for instructions. If you do not have a cluster environment (e.g. OpenShift) yet, grab the cluster-openshift.sh convenience script and run it to quickly start an OpenShift cluster environment with Istio installed. If you do not have a service mesh to test with, grab the install-bookinfo-demo.sh convenience script to download and install the Istio Bookinfo Demo service mesh.

Secrets

Note: There is a secret variable in the configuration called TOKEN_SECRET, be careful with this secret.

Running on OpenShift

Setting up OpenShift

The following section assumes that the user has OpenShift Origin installed.

The OpenShift Origin Documentation will outline all the steps required.

Building the Docker Image

Create the Kiali docker image through the "docker-build" make target:

cd ${GOPATH}/src/github.com/kiali/kiali
make docker-build

Deploying Kiali to OpenShift

Note
Before deploying and running Kiali, you must first install and deploy Istio. Required Istio Version: 1.0. There are a few places that you can reference in order to learn how to do this such as here, here, and here.
Note
The following make targets assume that the oc command is available in the user’s PATH and that the user is logged in. If you have istiooc instead, create a symlink in your PATH pointing oc to your istiooc binary.

The deploy and undeploy commands are automated in the Makefile. The following will undeploy an old installation of Kiali, if available, and deploy a new one:

make openshift-deploy

If you need to set the JAEGER or GRAFANA services, set the URL in the environment variable JAEGER_URL and GRAFANA_URL

Deploying Kiali to OpenShift with commands

Undeploying Kiali from OpenShift

If you want to remove Kiali from your OpenShift environment, you can do so by running the following command:

make openshift-undeploy

Reloading Kiali image in OpenShift

If you already have Kiali installed but you want to recreate the pod with a new docker image, you can run the following command:

make openshift-reload-image

Running on Kubernetes

Setting up Kubernetes

The following section assumes that the user has Kubernetes installed.

Building the Docker Image

Create the Kiali docker image through the "docker-build" make target:

cd ${GOPATH}/src/github.com/kiali/kiali
make docker-build

Note that if you are using minikube, you can build the docker image and push it directly into the minikube docker daemon using the alternative make target minikube-docker:

cd ${GOPATH}/src/github.com/kiali/kiali
make minikube-docker

Deploying Kiali to Kubernetes

Note
Before deploying and running Kiali, you must first install and deploy Istio. Required Istio Version: 1.0. There are a few places that you can reference in order to learn how to do this such as here, here, and here.
Note
The following make targets assume that the kubectl command is available in the user’s PATH.
Note
In order to deploy on Kubernetes and to be able to access the deployed service, you must ensure you have Ingress support. If you are using minikube, you need to run minikube addons enable ingress and add kiali as a hostname in your /etc/hosts via something like this command: echo "$(minikube ip) kiali" | sudo tee -a /etc/hosts

The deploy and undeploy commands are automated in the Makefile. The following will undeploy an old installation of Kiali, if available, and deploy a new one:

make k8s-deploy

If you need to set the JAEGER or GRAFANA services, set the URL in the environment variable JAEGER_URL and GRAFANA_URL

Undeploying Kiali from Kubernetes

If you want to remove Kiali from your Kubernetes environment, you can do so by running the following command:

make k8s-undeploy

Reloading Kiali image in Kubernetes

If you already have Kiali installed but you want to recreate the pod with a new docker image, you can run the following command:

make k8s-reload-image

Running Standalone

Sometimes you may want to run Kiali outside of any container environment, perhaps for debugging purposes. To do this, run:

cd ${GOPATH}/src/github.com/kiali/kiali
make install
make run

The "install" target installs the Kiali executable in your GOPATH /bin directory so you can run it outside of the Makefile:

cd ${GOPATH}/src/github.com/kiali/kiali
make install
${GOPATH}/bin/kiali -config <your-config-file>

Environment Variables

Many configuration settings can optionally be set via environment variables. If one of the environment variables below are set, they serve as the default value for its associated YAML configuration setting. The following are currently supported:

Environment Variable Name Description and YAML Setting

IDENTITY_CERT_FILE

Certificate file used to identify the file server. If set, you must go over https to retrieve content from the file server.

identity:
  cert_file: VALUE

IDENTITY_PRIVATE_KEY_FILE

Private key file used to identify the server. If set, you must go over https to retrieve content from the file server.

identity:
  private_key_file: VALUE

ISTIO_NAMESPACE

The namespace where Istio is installed. (Default: istio-system)

istio_namespace: VALUE

ISTIO_LABEL_NAME_APP

The label name of app required by Istio. (Default: app)

istio_labels:
  app_label_name: VALUE

ISTIO_LABEL_NAME_VERSION

The label name of version required by Istio. (Default: version)

istio_labels:
  version_label_name: VALUE

SERVER_ADDRESS

Where the http server is bound to.

server:
  address: VALUE

SERVER_PORT

Where the http server is listening.

server:
  port: VALUE

SERVER_CREDENTIALS_USERNAME

If this (and password) is set, these credentials are required in order to get content from the server.

server:
  credentials:
    username: VALUE

SERVER_CREDENTIALS_PASSWORD

If this (and username) is set, these credentials are required in order to get content from the server.

server:
  credentials:
    password: VALUE

SERVER_WEB_ROOT

Context root path to serve Kiali API and webapp from. (Default: /)

server:
  web_root: /VALUE

SERVER_CORS_ALLOW_ALL

When true, allows the web console to send requests to other domains other than where the console came from. Typically used for development environments only.

server:
  cors_allow_all: (true|false)

SERVER_STATIC_CONTENT_ROOT_DIRECTORY

The file server will serve all static content found under this root directory.

server:
  static_content_root_directory: VALUE

IN_CLUSTER

The annotation used by Istio in a Deployment template. If in_cluster is false then you need to set environments: KUBERNETES_SERVICE_HOST, KUBERNETES_SERVICE_PORT (Local development mode : oc proxy --port KUBERNETES_SERVICE_PORT ). (default is true)

in_cluster: (true|false)

PROMETHEUS_SERVICE_URL

The URL used to access and query the Prometheus Server. It must be accessible from Kiali pod. (default is http://prometheus.istio-system:9090)

external_services:
  prometheus_service_url: VALUE

ISTIO_SIDECAR_ANNOTATION

The annotation used by Istio in Pods. (default is sidecar.istio.io/status)

external_services:
  istio:
    istio_sidecar_annotation: VALUE

ISTIO_IDENTITY_DOMAIN

The annotation used by Istio how Identity Domain. (default is svc.cluster.local)

external_services:
  istio:
    istio_identity_domain: VALUE

ISTIO_URL_SERVICE_VERSION

The Service of Istio to check version. (default is http://istio-pilot:9093/version)

external_services:
  istio:
    istio_url_service_version: VALUE

GRAFANA_DISPLAY_LINK

When true, a link to Grafana will be displayed for more metrics.

external_services:
  grafana:
    display_link: (true|false)

GRAFANA_URL

The URL to the Grafana service. When not set, Kiali throw an error when the user try request to Grafana (/api/grafana).

external_services:
  grafana:
    url: VALUE

GRAFANA_SERVICE_NAMESPACE

The Kubernetes namespace that holds the Grafana service. This configuration is ignored if GRAFANA_URL is set. (default is istio-system)

external_services:
  grafana:
    service_namespace: VALUE

GRAFANA_SERVICE

The OpenShift route name or the Kubernetes service name for Grafana. This configuration is ignored if GRAFANA_URL is set. (default is grafana)

external_services:
  grafana:
    service: VALUE

GRAFANA_SERVICE_DASHBOARD_PATTERN

Search pattern for Grafana Service dashboard. (default is Istio%20Service%20Dashboard)

external_services:
  grafana:
    service_dashboard_pattern: VALUE

GRAFANA_WORKLOAD_DASHBOARD_PATTERN

Search pattern for Grafana Workload dashboard. (default is Istio%20Workload%20Dashboard)

external_services:
  grafana:
    workload_dashboard_pattern: VALUE

GRAFANA_VAR_NAMESPACE

The name of the Grafana variable that controls namespaces in dashboards. (default is var-namespace)

external_services:
  grafana:
    var_namespace: VALUE

GRAFANA_VAR_SERVICE

The name of the Grafana variable that controls services in dashboards. (default is var-service)

external_services:
  grafana:
    var_service: VALUE

GRAFANA_VAR_WORKLOAD

The name of the Grafana variable that controls workloads in dashboards. (default is var-workload)

external_services:
  grafana:
    var_workload: VALUE

JAEGER_URL

The URL to the Jaeger service. When not set, Kiali throw an error when the user try request to Jaeger (/api/jaeger).

external_services:
  jaeger:
    url: VALUE

API_NAMESPACES_EXCLUDE

An optional list of namespaces/projects excluded from the list of namespaces provided by the API and UI. Regex is supported. This does not affect explicit namespace access.

api:
  namespaces:
    exclude:
    - namespacePattern1
    - namespacePattern2
    - etc..

LOGIN_TOKEN_SIGNING_KEY

The signing key used to generate tokens for user authentication. (default is kiali)

login_token:
  signing_key: VALUE

LOGIN_TOKEN_EXPIRATION_SECONDS

The token expiration in seconds. (default is 10 hours ⇒ 36000)

login_token:
  expiration_seconds: VALUE

KUBERNETES_BURST

The Burst value of Kubernetes client (default is 200)

kubernetes_config:
  burst: VALUE

KUBERNETES_QPS

The QPS value of Kubernetes client (default is 175)

kubernetes_config:
  qps: VALUE

KUBERNETES_CACHE_ENABLED

Flag to use a Kubernetes cache for watching changes and updating pods and controllers data asynchronously.

Important

Kubernetes cache is not compatible with reduced permissions scenearios.

(default is false)

kubernetes_config:
  cache_enabled: VALUE

KUBERNETES_CACHE_DURATION

The ratio interval (expressed in nanoseconds) used for the cache to perform a full refresh.

(default is 300000000)

kubernetes_config:
  cache_duration: VALUE

Configure External Services

Jaeger

To configure the Jaeger Service you need to expose the service and set the URL parameter in the config.yaml

external_services:
  jaeger:
    url: https://jaeger-query-istio-system.127.0.0.1.nip.io

If you are using the Make task openshift-deploy or k8s-deploy you need to change in the kiali-configmap.yaml the value of the jaeger > url

apiVersion: v1
kind: ConfigMap
metadata:
  name: kiali
  labels:
    app: kiali
    version: ${VERSION_LABEL}
data:
  config.yaml: |
    server:
      port: 20001
      static_content_root_directory: /opt/kiali/console
    external_services:
      jaeger:
        url: http://jaeger-query-istio-system.127.0.0.1.nip.io
      grafana:
        url: http://grafana-istio-system.127.0.0.1.nip.io

Grafana

To configure the Grafana Service you need to expose the service and set the URL parameter in the config.yaml

external_services:
  grafana:
    url: http://grafana-istio-system.127.0.0.1.nip.io

If you are using the Make task openshift-deploy or k8s-deploy you need to change in the kiali-configmap.yaml the value of the grafana > url

apiVersion: v1
kind: ConfigMap
metadata:
  name: kiali
  labels:
    app: kiali
    version: ${VERSION_LABEL}
data:
  config.yaml: |
    server:
      port: 20001
      static_content_root_directory: /opt/kiali/console
    external_services:
      jaeger:
        url: http://jaeger-query-istio-system.127.0.0.1.nip.io
      grafana:
        url: http://grafana-istio-system.127.0.0.1.nip.io

Additional Notes

Customize the UI web context root

By default Kiali UI is deployed to the top level of https://kiali-istio-system.<your_cluster_domain_or_ip>/. In some situation such as when you want to serve Kiali UI along with other apps under the same host name, e.g., example.com/kiali, example.com/app1, you can edit Kiali Config Map and provide a different value for web_root. Note: the path must begin with a /.

An example of custom web root:

server:
  port: 20001
  web_root: /kiali
external_services:
  jaeger:
  ...

Running the UI Outside the Core

When developing the Kiali UI you will find it useful to run it outside of the core to make it easier to update the UI code and see the changes without having to recompile. The prefered approach for this is to use a proxy on the UI to mount the core. The process is described here.

To connect with the backend and avoid the javascript prompt requesting authentication you need to send the requests with a specific header.

X-Auth-Type-Kiali-UI: 1

The response will contain the header

WWW-Authenticate: xBasic realm="Kiali"

Otherwise the header will be

WWW-Authenticate: Basic realm="Kiali"

Running A Locally Built UI Inside the Core

If you are developing the UI on your local machine but you want to see it deployed and running inside of the core server, you can do so by setting the environment variable CONSOLE_VERSION to the value "local" when building the docker image via the docker-build target. By default, your UI’s build/ directory is assumed to be in a directory called kiali-ui that is a peer directory of the GOPATH root directory for the core server. If it is not, you can set the environment variable CONSOLE_LOCAL_DIR to the value of the path of the root directory for the UI such that $CONSOLE_LOCAL_DIR/build contains the generated build files for the UI.

For example, if your GOPATH directory for the Kiali project is /source/kiali/kiali and you have git cloned the Kiali UI repository in /source/kiali/kiali-ui then you do not need to set CONSOLE_LOCAL_DIR. You can embed your locally built console into the core docker image via:

CONSOLE_VERSION=local make docker-build

If you git cloned the Kiali UI repository in directory /my/git/repo and have built the UI there (such that the build files are located at /my/git/repo/build) then you can embed that locally built console into the core docker image via:

CONSOLE_VERSION=local CONSOLE_LOCAL_DIR=/my/git/repo make docker-build

Disabling SSL

In the provided OpenShift templates, SSL is turned on by default. If you want to turn it off, you should: * Remove the "tls: termination: reencrypt" option in Kiali route * Remove the "identity" block, with certificate paths, in Kiali Config Map. * Optionnally you can also remove the annotation "service.alpha.openshift.io/serving-cert-secret-name", and the related volume that is declared and mounted in Kiali Deployment (but if you don’t, they will just be ignored).

Running Kiali with commands

Download the files and configure

Note
Before deploying and running Kiali, you must first install and deploy Istio. Required Istio Version: 1.0. There are a few places that you can reference in order to learn how to do this such as here, here, and here.
Note
The following make targets assume that the oc command is available in the user’s PATH and that the user is logged in. If you have istiooc instead, create a symlink in your PATH pointing oc to your istiooc binary.
Get the kiali-configmap.yaml
$ curl https://raw.githubusercontent.com/kiali/kiali/master/deploy/kubernetes/kiali-configmap.yaml | \
VERSION_LABEL=master envsubst > kiali-configmap.yaml

If you need to set the JAEGER and GRAFANA services set the URL configuration.

$ curl https://raw.githubusercontent.com/kiali/kiali/master/deploy/kubernetes/kiali-configmap.yaml | \
VERSION_LABEL=master JAEGER_URL=http://jaeger-query-istio-system.127.0.0.1.nip.io GRAFANA_URL=http://grafana-istio-system.127.0.0.1.nip.io envsubst > kiali-configmap.yaml
Get the kiali-secrets.yaml
$ curl https://raw.githubusercontent.com/kiali/kiali/master/deploy/kubernetes/kiali-secrets.yaml | \
VERSION_LABEL=master envsubst > kiali-secrets.yaml
Get the kiali.yaml
curl https://raw.githubusercontent.com/kiali/kiali/master/deploy/kubernetes/kiali.yaml | \
IMAGE_NAME=kiali/kiali \
IMAGE_VERSION=latest \
NAMESPACE=istio-system \
VERSION_LABEL=master \
VERBOSE_MODE=4 envsubst > kiali.yaml

Running in Openshift

oc create -f kiali-configmap.yaml -n istio-system
oc create -f kiali-secrets.yaml -n istio-system
oc create -f kiali.yaml -n istio-system

Running in kubernetes

kubectl create -f kiali-configmap.yaml -n istio-system
kubectl create -f kiali-secrets.yaml -n istio-system
kubectl create -f kiali.yaml -n istio-system

Contributing

All contributions are welcome - Kiali uses the Apache 2 license and does not require any contributor agreement to submit patches. To setup your environment, check instructions in the Building and Running sections.

Get involved by submitting pull requests on GitHub. Before you do, please talk to us.

Talk to us

Ask questions on the Kiali IRC channel (#kiali on freenode) or the Google Groups: kiali-users or kiali-dev.

Issue tracking

The Kiali team is using JIRA for issue tracking. If you do not have a JIRA account, you can also open issues here on GitHub (we are monitoring this as well) for any bugs or problems you encounter or to suggest new features.

When you are looking for issues to get started, you can use this JIRA query for good first issues. If you pick one from the list, please let us know by the above mentioned means.

Code Style Guide