Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate BanyanDB as storage solution #126

Merged
merged 21 commits into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions .github/workflows/e2e.ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ on:
env:
SKIP_TEST: true
ISTIO_VERSION: 1.7.1
OAP_TAG: cef4f6dcac5b14cd67f51a64ad08052bbc046653
OAP_TAG: 14fbd975be077082ed6fa604e05479dd9af0d995
OAP_REPO: ghcr.io/apache/skywalking/oap
UI_TAG: cef4f6dcac5b14cd67f51a64ad08052bbc046653
UI_TAG: 14fbd975be077082ed6fa604e05479dd9af0d995
UI_REPO: ghcr.io/apache/skywalking/ui
SATELLITE_TAG: v64f1938e118e95b8d8f50756f666e4777ce6aca2
SATELLITE_TAG: v35bfaff6352b4dc351a706772796a1f79b651c14
SATELLITE_REPO: ghcr.io/apache/skywalking-satellite/skywalking-satellite

jobs:
Expand All @@ -43,28 +43,33 @@ jobs:
images:
- name: recent
oap-repo: ghcr.io/apache/skywalking/oap
oap-tag: cef4f6dcac5b14cd67f51a64ad08052bbc046653
oap-tag: 14fbd975be077082ed6fa604e05479dd9af0d995
ui-repo: ghcr.io/apache/skywalking/ui
ui-tag: cef4f6dcac5b14cd67f51a64ad08052bbc046653
ui-tag: 14fbd975be077082ed6fa604e05479dd9af0d995
name: Istio+Envoy Access Log Service(${{ matrix.images.name }})
env:
OAP_TAG: ${{ matrix.images.oap-tag }}
OAP_REPO: ${{ matrix.images.oap-repo }}
UI_TAG: ${{ matrix.images.ui-tag }}
UI_REPO: ${{ matrix.images.ui-repo }}
SATELLITE_TAG: v64f1938e118e95b8d8f50756f666e4777ce6aca2
SATELLITE_TAG: v35bfaff6352b4dc351a706772796a1f79b651c14
SATELLITE_REPO: ghcr.io/apache/skywalking-satellite/skywalking-satellite
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- name: Setup go
uses: actions/setup-go@v2
uses: actions/setup-go@v4
with:
go-version: '1.16'
- name: Run Skywalking E2E Test
go-version: '1.20'
- run: go version
- name: Run Skywalking E2E Test (Elasticsearch as database)
uses: apache/skywalking-infra-e2e@45584853d6f660102c523b1e9cb5815d12ae55d3
with:
e2e-file: $GITHUB_WORKSPACE/test/e2e/e2e.yaml
e2e-file: $GITHUB_WORKSPACE/test/e2e/e2e-elasticsearch.yaml
- name: Run Skywalking E2E Test (BanyanDB as database)
uses: apache/skywalking-infra-e2e@45584853d6f660102c523b1e9cb5815d12ae55d3
with:
e2e-file: $GITHUB_WORKSPACE/test/e2e/e2e-banyandb.yaml
- name: Run SWCK E2E Test
uses: apache/skywalking-infra-e2e@afdf1cca0519d65bc480d8680b7a27f9b41fc421
with:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should run e2e in the parallel mode. You could ref the main repo skywalking.yml to use matrix to define name and config.

Expand Down
6 changes: 6 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@ Changes by Version
==================
Release Notes.

4.6.0
------------------

- Integrate BanyanDB as storage solution
- Bump up swck to v0.8.0

4.5.0
------------------

Expand Down
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ There are required values that you must set explicitly when deploying SkyWalking
| name | description | example |
| ---- | ----------- | ------- |
| `oap.image.tag` | the OAP docker image tag | `9.2.0` |
| `oap.storageType` | the storage type of the OAP | `elasticsearch`, `postgresql`, etc. |
| `oap.storageType` | the storage type of the OAP | `elasticsearch`, `postgresql`, `banyandb`, etc. |
| `ui.image.tag` | the UI docker image tag | `9.2.0` |

You can set these required values via command line (e.g. `--set oap.image.tag=9.2.0 --set oap.storageType=elasticsearch`),
Expand Down Expand Up @@ -66,6 +66,22 @@ helm repo add elastic https://helm.elastic.co
helm dep up ${REPO}/skywalking
```

To use banyandb as storage solution, you can try

```shell
export REPO=chart
git clone https://github.com/apache/skywalking-kubernetes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
git clone https://github.com/apache/skywalking-kubernetes
git clone https://github.com/apache/skywalking-helm

cd skywalking-kubernetes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cd skywalking-kubernetes
cd skywalking-helm

helm install "${SKYWALKING_RELEASE_NAME}" \
${REPO}/skywalking \
-n "${SKYWALKING_RELEASE_NAMESPACE}" \
--set oap.image.tag=9.5.0 \
--set oap.storageType=banyandb \
--set ui.image.tag=9.5.0 \
--set elasticsearch.enabled=false \
--set banyandb.enabled=true
```

## Install development version of SWCK Adapter using master branch

This is needed **only** when you want to install [SWCK Adapter](https://github.com/apache/skywalking-swck/tree/master/adapter) from master branch.
Expand Down Expand Up @@ -136,6 +152,7 @@ elasticsearch:
user: "xxx" # [optional]
password: "xxx" # [optional]
```
The same goes for PostgreSQL and BanyanDB.

## Install development version using ghcr.io Helm repository

Expand All @@ -161,9 +178,9 @@ This is needed **only** when you want to install source codes.
helm install "${SKYWALKING_RELEASE_NAME}" ${REPO}/skywalking -n "${SKYWALKING_RELEASE_NAMESPACE}"
```

## Install a specific version of SkyWalking with an existing Elasticsearch
## Install a specific version of SkyWalking with an existing database

Modify the connection information to the existing elasticsearch cluster in file [`values-my-es.yaml`](chart/skywalking/values-my-es.yaml).
If you want to use a specific version of elasticsearch as storage solution, for instance, modify the connection information to the existing elasticsearch cluster in file [`values-my-es.yaml`](chart/skywalking/values-my-es.yaml).

```shell script
helm install "${SKYWALKING_RELEASE_NAME}" ${REPO}/skywalking -n "${SKYWALKING_RELEASE_NAMESPACE}" \
Expand Down
2 changes: 1 addition & 1 deletion chart/adapter/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
apiVersion: v2
name: skywalking-helm-swck-adapter
home: https://github.com/apache/skywalking-swck/tree/master/adapter
version: 0.7.0
version: 0.8.0
description: Provide custom metrics coming from SkyWalking OAP cluster for autoscaling by Kubernetes HPA
icon: https://raw.githubusercontent.com/apache/skywalking-kubernetes/master/logo/sw-logo-for-chart.jpg
sources:
Expand Down
2 changes: 1 addition & 1 deletion chart/adapter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ The following table lists the configurable parameters of the adapter chart and t
| `.serviceAccountName` | The service account name of adapter | `skywalking-custom-metrics-apiserver` |
| `.image.repository` | Adapter container image name | `docker.io/apache/skywalking-swck` |
| `.image.pullPolicy` | Adapter container image pull policy | `IfNotPresent` |
| `.image.tag` | Adapter container image tag | `v0.7.0` |
| `.image.tag` | Adapter container image tag | `v0.8.0` |
| `.service.port` | The port for the adapter service | `6.1.0` |
| `.oap.service.name` | The service name of OAP | `skywalking-system-oap` |
| `.oap.service.namespace` | The service namespace of OAP | `skywalking-system` |
Expand Down
2 changes: 1 addition & 1 deletion chart/adapter/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ serviceAccountName: skywalking-custom-metrics-apiserver
image:
repository: docker.io/apache/skywalking-swck
pullPolicy: IfNotPresent
tag: "v0.7.0"
tag: "v0.8.0"
service:
port: 6443
oap:
Expand Down
2 changes: 1 addition & 1 deletion chart/operator/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
apiVersion: v2
name: skywalking-helm-swck-operator
home: https://github.com/apache/skywalking-swck/tree/master/operator
version: 0.7.0
version: 0.8.0
description: Provision and maintain SkyWalking backend components
icon: https://raw.githubusercontent.com/apache/skywalking-kubernetes/master/logo/sw-logo-for-chart.jpg
sources:
Expand Down
2 changes: 1 addition & 1 deletion chart/operator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The following table lists the configurable parameters of the operator chart and
| `.serviceAccountName` | The service account name of operator | `skywalking-swck-controller-manager` |
| `.image.repository` | Operator container image name | `docker.io/apache/skywalking-swck` |
| `.image.pullPolicy` | Operator container image pull policy | `IfNotPresent` |
| `.image.tag` | Operator container image tag | `v0.7.0` |
| `.image.tag` | Operator container image tag | `v0.8.0` |
| `.metrics.service.port` | The port for the operator metrics service | `8443` |
| `.webhook.service.port` | The port for the operator web hook service | `9443` |
| `.resources.limits.cpu` | The limits of cpu in the operator | `200m` |
Expand Down
18 changes: 9 additions & 9 deletions chart/operator/templates/crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: fetchers.operator.skywalking.apache.org
spec:
conversion:
Expand Down Expand Up @@ -131,7 +131,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: javaagents.operator.skywalking.apache.org
spec:
conversion:
Expand Down Expand Up @@ -237,7 +237,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: oapserverconfigs.operator.skywalking.apache.org
spec:
conversion:
Expand Down Expand Up @@ -453,7 +453,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
creationTimestamp: null
name: oapserverdynamicconfigs.operator.skywalking.apache.org
spec:
Expand Down Expand Up @@ -550,7 +550,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: oapservers.operator.skywalking.apache.org
spec:
conversion:
Expand Down Expand Up @@ -1118,7 +1118,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: satellites.operator.skywalking.apache.org
spec:
conversion:
Expand Down Expand Up @@ -1463,7 +1463,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: storages.operator.skywalking.apache.org
spec:
conversion:
Expand Down Expand Up @@ -1721,7 +1721,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
creationTimestamp: null
name: swagents.operator.skywalking.apache.org
spec:
Expand Down Expand Up @@ -1981,7 +1981,7 @@ kind: CustomResourceDefinition
metadata:
annotations:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "operator.fullname" . | trunc (int (sub 63 (len "-serving-cert"))) | printf "%s-serving-cert" }}
controller-gen.kubebuilder.io/version: v0.7.0
controller-gen.kubebuilder.io/version: v0.8.0
name: uis.operator.skywalking.apache.org
spec:
conversion:
Expand Down
2 changes: 1 addition & 1 deletion chart/operator/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ serviceAccountName: skywalking-swck-controller-manager
image:
repository: docker.io/apache/skywalking-swck
pullPolicy: IfNotPresent
tag: "v0.7.0"
tag: "v0.8.0"
securityContext:
allowPrivilegeEscalation: false
metrics:
Expand Down
4 changes: 4 additions & 0 deletions chart/skywalking/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,7 @@ dependencies:
version: 12.1.2
repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
condition: postgresql.enabled
- name: skywalking-banyandb-helm
version: 0.0.0-67df1c7
repository: oci://ghcr.io/apache/skywalking-banyandb-helm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we could use this as the final release. We should request for docker hub repository, and do a release for banyandb helm. @hanahmily

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we deploy skywalking chart snapshot to ghcr so this makes sense in main branch but have to remember to change to a released one for banyandb chart before releasing. This also implies that a skywalking chart release depends on a banyandb release.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. We will release the banyandb helm once the cluster mode is supported later. @ButterBright please make a note of this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we deploy skywalking chart snapshot to ghcr so this makes sense in main branch but have to remember to change to a released one for banyandb chart before releasing. This also implies that a skywalking chart release depends on a banyandb release.

For me, the concern is, that we don't release helm repo in high frequency, so, most likely we definitely will lose track and release the next helm with this snapshot repository.

This PR is never urgent, so, if you feel the BanyanDB helm repo is ready to release, please go ahead to do so. I believe no matter banyandb helm repo is relying on BanyanDB server 0.5 release or not, it is close to be ready.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. We will release the banyandb helm once the cluster mode is supported later. @ButterBright please make a note of this.

Okay.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we deploy skywalking chart snapshot to ghcr so this makes sense in main branch but have to remember to change to a released one for banyandb chart before releasing. This also implies that a skywalking chart release depends on a banyandb release.

For me, the concern is, that we don't release helm repo in high frequency, so, most likely we definitely will lose track and release the next helm with this snapshot repository.

This PR is never urgent, so, if you feel the BanyanDB helm repo is ready to release, please go ahead to do so. I believe no matter banyandb helm repo is relying on BanyanDB server 0.5 release or not, it is close to be ready.

Let's wait for banyandb's first official release then.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. @ButterBright, could you please release the banyandb helm chart version 0.1.0? @kezhenxu94, would you kindly guide them through the process?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. @ButterBright, could you please release the banyandb helm chart version 0.1.0? @kezhenxu94, would you kindly guide them through the process?

Reach out to me if there is any help I can offer, btw, it's too troublesome to ask a non-committer to release, even it's a committer who is releasing, one of the steps in release process still needs a PMC member to help, so basically there is few stuff can be done by @ButterBright to propose a release yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't accept non committer to do a release. Because ASF requires apache ID and his/her sign based on that.
@hanahmily You need to take the responsibility to be release manager.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will take care of the release

condition: banyandb.enabled
19 changes: 19 additions & 0 deletions chart/skywalking/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,16 @@ Create the name of the service account to use for the satellite cluster
echo "Waiting for postgresql..."
sleep 3
done
{{- else if eq .Values.oap.storageType "banyandb" -}}
{{- $banyandbHost := "banyandb-http" -}}
{{- if not .Values.banyandb.enabled -}}
{{- $banyandbHost = .Values.banyandb.config.httpHost -}}
{{- end }}
- name: wait-for-banyandb
image: curlimages/curl
imagePullPolicy: IfNotPresent
command: ['sh', '-c', 'for i in $(seq 1 60); do curl {{ $banyandbHost }}:{{ .Values.banyandb.config.httpPort }}/api/healthz && exit 0 || sleep 5; done; exit 1']

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

{{- end }}
{{- end -}}

Expand Down Expand Up @@ -150,5 +160,14 @@ Create the name of the service account to use for the satellite cluster
value: "{{ .Values.postgresql.auth.username }}"
- name: SW_DATA_SOURCE_PASSWORD
value: "{{ .Values.postgresql.auth.password }}"
{{- else if eq .Values.oap.storageType "banyandb" }}
{{- $banyandbHost := "banyandb-grpc" -}}
{{- if not .Values.banyandb.enabled -}}
{{- $banyandbHost = .Values.banyandb.config.grpcHost -}}
{{- end }}
- name: SW_STORAGE_BANYANDB_HOST
value: "{{ $banyandbHost }}"
- name: SW_STORAGE_BANYANDB_PORT
value: "{{ .Values.banyandb.config.grpcPort }}"
{{- end }}
{{- end -}}
8 changes: 8 additions & 0 deletions chart/skywalking/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -444,6 +444,14 @@ postgresql:
persistence:
enabled: false

banyandb:
enabled: false
config:
grpcHost: banyandb-grpc
httpHost: banyandb-http
grpcPort: 17912
httpPort: 17913

satellite:
name: satellite
replicas: 1
Expand Down
Loading
Loading