diff --git a/infra/charts/feast/README.md b/infra/charts/feast/README.md index ab5321ca865..e93b687f191 100644 --- a/infra/charts/feast/README.md +++ b/infra/charts/feast/README.md @@ -81,17 +81,26 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-core.kafka.topics[0].name` | Default topic name in Kafka| `feast` | `feast-core.kafka.topics[0].replicationFactor` | No of replication factor for the topic| `1` | `feast-core.kafka.topics[0].partitions` | No of partitions for the topic | `1` +| `feast-core.prometheus-statsd-exporter.enabled` | Flag to install Prometheus StatsD Exporter | `false` +| `feast-core.prometheus-statsd-exporter.*` | Refer to this [link](charts/feast-core/charts/prometheus-statsd-exporter/values.yaml | | `feast-core.replicaCount` | No of pods to create | `1` | `feast-core.image.repository` | Repository for Feast Core Docker image | `gcr.io/kf-feast/feast-core` -| `feast-core.image.tag` | Tag for Feast Core Docker image | `0.3.2` +| `feast-core.image.tag` | Tag for Feast Core Docker image | `0.4.4` | `feast-core.image.pullPolicy` | Image pull policy for Feast Core Docker image | `IfNotPresent` +| `feast-core.prometheus.enabled` | Add annotations to enable Prometheus scraping | `false` | `feast-core.application.yaml` | Configuration for Feast Core application | Refer to this [link](charts/feast-core/values.yaml) | `feast-core.springConfigMountPath` | Directory to mount application.yaml | `/etc/feast/feast-core` | `feast-core.gcpServiceAccount.useExistingSecret` | Flag to use existing secret for GCP service account | `false` | `feast-core.gcpServiceAccount.existingSecret.name` | Secret name for the service account | `feast-gcp-service-account` | `feast-core.gcpServiceAccount.existingSecret.key` | Secret key for the service account | `key.json` | `feast-core.gcpServiceAccount.mountPath` | Directory to mount the JSON key file | `/etc/gcloud/service-accounts` +| `feast-core.gcpProjectId` | Project ID to set `GOOGLE_CLOUD_PROJECT` to change default project used by SDKs | `""` +| `feast-core.jarPath` | Path to Jar file in the Docker image | `/opt/feast/feast-core.jar` | `feast-core.jvmOptions` | Options for the JVM | `[]` +| `feast-core.logLevel` | Application logging level | `warn` +| `feast-core.logType` | Application logging type (`JSON` or `Console`) | `JSON` +| `feast-core.springConfigProfiles` | Map of profile name to file content for additional Spring profiles | `{}` +| `feast-core.springConfigProfilesActive` | CSV of profiles to enable from `springConfigProfiles` | `""` | `feast-core.livenessProbe.enabled` | Flag to enable liveness probe | `true` | `feast-core.livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `60` | `feast-core.livenessProbe.periodSeconds` | How often to perform the probe | `10` @@ -109,6 +118,7 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-core.grpc.port` | Kubernetes Service port for GRPC request| `6565` | `feast-core.grpc.targetPort` | Container port for GRPC request| `6565` | `feast-core.resources` | CPU and memory allocation for the pod | `{}` +| `feast-core.ingress` | See *Ingress Parameters* [below](#ingress-parameters) | `{}` | `feast-serving-online.enabled` | Flag to install Feast Online Serving | `true` | `feast-serving-online.redis.enabled` | Flag to install Redis in Feast Serving | `false` | `feast-serving-online.redis.usePassword` | Flag to use password to access Redis | `false` @@ -116,8 +126,9 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-serving-online.core.enabled` | Flag for Feast Serving to use Feast Core in the same Helm release | `true` | `feast-serving-online.replicaCount` | No of pods to create | `1` | `feast-serving-online.image.repository` | Repository for Feast Serving Docker image | `gcr.io/kf-feast/feast-serving` -| `feast-serving-online.image.tag` | Tag for Feast Serving Docker image | `0.3.2` +| `feast-serving-online.image.tag` | Tag for Feast Serving Docker image | `0.4.4` | `feast-serving-online.image.pullPolicy` | Image pull policy for Feast Serving Docker image | `IfNotPresent` +| `feast-serving-online.prometheus.enabled` | Add annotations to enable Prometheus scraping | `true` | `feast-serving-online.application.yaml` | Application configuration for Feast Serving | Refer to this [link](charts/feast-serving/values.yaml) | `feast-serving-online.store.yaml` | Store configuration for Feast Serving | Refer to this [link](charts/feast-serving/values.yaml) | `feast-serving-online.springConfigMountPath` | Directory to mount application.yaml and store.yaml | `/etc/feast/feast-serving` @@ -125,7 +136,13 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-serving-online.gcpServiceAccount.existingSecret.name` | Secret name for the service account | `feast-gcp-service-account` | `feast-serving-online.gcpServiceAccount.existingSecret.key` | Secret key for the service account | `key.json` | `feast-serving-online.gcpServiceAccount.mountPath` | Directory to mount the JSON key file | `/etc/gcloud/service-accounts` +| `feast-serving-online.gcpProjectId` | Project ID to set `GOOGLE_CLOUD_PROJECT` to change default project used by SDKs | `""` +| `feast-serving-online.jarPath` | Path to Jar file in the Docker image | `/opt/feast/feast-serving.jar` | `feast-serving-online.jvmOptions` | Options for the JVM | `[]` +| `feast-serving-online.logLevel` | Application logging level | `warn` +| `feast-serving-online.logType` | Application logging type (`JSON` or `Console`) | `JSON` +| `feast-serving-online.springConfigProfiles` | Map of profile name to file content for additional Spring profiles | `{}` +| `feast-serving-online.springConfigProfilesActive` | CSV of profiles to enable from `springConfigProfiles` | `""` | `feast-serving-online.livenessProbe.enabled` | Flag to enable liveness probe | `true` | `feast-serving-online.livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `60` | `feast-serving-online.livenessProbe.periodSeconds` | How often to perform the probe | `10` @@ -143,6 +160,7 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-serving-online.grpc.port` | Kubernetes Service port for GRPC request| `6566` | `feast-serving-online.grpc.targetPort` | Container port for GRPC request| `6566` | `feast-serving-online.resources` | CPU and memory allocation for the pod | `{}` +| `feast-serving-online.ingress` | See *Ingress Parameters* [below](#ingress-parameters) | `{}` | `feast-serving-batch.enabled` | Flag to install Feast Batch Serving | `true` | `feast-serving-batch.redis.enabled` | Flag to install Redis in Feast Serving | `false` | `feast-serving-batch.redis.usePassword` | Flag to use password to access Redis | `false` @@ -150,8 +168,9 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-serving-batch.core.enabled` | Flag for Feast Serving to use Feast Core in the same Helm release | `true` | `feast-serving-batch.replicaCount` | No of pods to create | `1` | `feast-serving-batch.image.repository` | Repository for Feast Serving Docker image | `gcr.io/kf-feast/feast-serving` -| `feast-serving-batch.image.tag` | Tag for Feast Serving Docker image | `0.3.2` +| `feast-serving-batch.image.tag` | Tag for Feast Serving Docker image | `0.4.4` | `feast-serving-batch.image.pullPolicy` | Image pull policy for Feast Serving Docker image | `IfNotPresent` +| `feast-serving-batch.prometheus.enabled` | Add annotations to enable Prometheus scraping | `true` | `feast-serving-batch.application.yaml` | Application configuration for Feast Serving | Refer to this [link](charts/feast-serving/values.yaml) | `feast-serving-batch.store.yaml` | Store configuration for Feast Serving | Refer to this [link](charts/feast-serving/values.yaml) | `feast-serving-batch.springConfigMountPath` | Directory to mount application.yaml and store.yaml | `/etc/feast/feast-serving` @@ -159,7 +178,13 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-serving-batch.gcpServiceAccount.existingSecret.name` | Secret name for the service account | `feast-gcp-service-account` | `feast-serving-batch.gcpServiceAccount.existingSecret.key` | Secret key for the service account | `key.json` | `feast-serving-batch.gcpServiceAccount.mountPath` | Directory to mount the JSON key file | `/etc/gcloud/service-accounts` +| `feast-serving-batch.gcpProjectId` | Project ID to set `GOOGLE_CLOUD_PROJECT` to change default project used by SDKs | `""` +| `feast-serving-batch.jarPath` | Path to Jar file in the Docker image | `/opt/feast/feast-serving.jar` | `feast-serving-batch.jvmOptions` | Options for the JVM | `[]` +| `feast-serving-batch.logLevel` | Application logging level | `warn` +| `feast-serving-batch.logType` | Application logging type (`JSON` or `Console`) | `JSON` +| `feast-serving-batch.springConfigProfiles` | Map of profile name to file content for additional Spring profiles | `{}` +| `feast-serving-batch.springConfigProfilesActive` | CSV of profiles to enable from `springConfigProfiles` | `""` | `feast-serving-batch.livenessProbe.enabled` | Flag to enable liveness probe | `true` | `feast-serving-batch.livenessProbe.initialDelaySeconds` | Delay before liveness probe is initiated | `60` | `feast-serving-batch.livenessProbe.periodSeconds` | How often to perform the probe | `10` @@ -176,4 +201,51 @@ The following table lists the configurable parameters of the Feast chart and the | `feast-serving-batch.http.targetPort` | Container port for HTTP request | `8080` | `feast-serving-batch.grpc.port` | Kubernetes Service port for GRPC request| `6566` | `feast-serving-batch.grpc.targetPort` | Container port for GRPC request| `6566` -| `feast-serving-batch.resources` | CPU and memory allocation for the pod | `{}` \ No newline at end of file +| `feast-serving-batch.resources` | CPU and memory allocation for the pod | `{}` +| `feast-serving-batch.ingress` | See *Ingress Parameters* [below](#ingress-parameters) | `{}` + +## Ingress Parameters + +The following table lists the configurable parameters of the ingress section for each Feast module. + +Note, there are two ingresses available for each module - `grpc` and `http`. + +| Parameter | Description | Default +| ----------------------------- | ----------- | ------- +| `ingress.grcp.enabled` | Enables an ingress (endpoint) for the gRPC server | `false` +| `ingress.grcp.*` | See below | +| `ingress.http.enabled` | Enables an ingress (endpoint) for the HTTP server | `false` +| `ingress.http.*` | See below | +| `ingress.*.class` | Value for `kubernetes.io/ingress.class` | `nginx` +| `ingress.*.hosts` | List of host-names for the ingress | `[]` +| `ingress.*.annotations` | Additional ingress annotations | `{}` +| `ingress.*.https.enabled` | Add a tls section to the ingress | `true` +| `ingress.*.https.secretNames` | Map of hostname to TLS secret name | `{}` If not specified, defaults to `domain-tld-tls` e.g. `feast.example.com` uses secret `example-com-tls` +| `ingress.*.auth.enabled` | Enable auth on the ingress (only applicable for `nginx` type | `false` +| `ingress.*.auth.signinHost` | External hostname of the OAuth2 proxy to use | First item in `ingress.hosts`, replacing the sub-domain with 'auth' e.g. `feast.example.com` uses `auth.example.com` +| `ingress.*.auth.authUrl` | Internal URI to internal auth endpoint | `http://auth-server.auth-ns.svc.cluster.local/auth` +| `ingress.*.whitelist` | Subnet masks to whitelist (i.e. value for `nginx.ingress.kubernetes.io/whitelist-source-range`) | `"""` + +To enable all the ingresses will a config like the following (while also adding the hosts etc): + +```yaml +feast-core: + ingress: + grpc: + enabled: true + http: + enabled: true +feast-serving-online: + ingress: + grpc: + enabled: true + http: + enabled: true +feast-serving-batch: + ingress: + grpc: + enabled: true + http: + enabled: true +``` + diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/.helmignore b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/.helmignore similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/.helmignore rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/.helmignore diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/Chart.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/Chart.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/Chart.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/Chart.yaml diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/README.md b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/README.md similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/README.md rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/README.md diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/NOTES.txt b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/NOTES.txt similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/NOTES.txt rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/NOTES.txt diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/_helpers.tpl b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/_helpers.tpl similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/_helpers.tpl rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/_helpers.tpl diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/config.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/config.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/config.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/config.yaml diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/deployment.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/deployment.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/deployment.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/deployment.yaml diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/pvc.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/pvc.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/pvc.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/pvc.yaml diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/service.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/service.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/service.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/service.yaml diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/templates/serviceaccount.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/serviceaccount.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/templates/serviceaccount.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/templates/serviceaccount.yaml diff --git a/infra/charts/feast/charts/prometheus-statsd-exporter/values.yaml b/infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/values.yaml similarity index 100% rename from infra/charts/feast/charts/prometheus-statsd-exporter/values.yaml rename to infra/charts/feast/charts/feast-core/charts/prometheus-statsd-exporter/values.yaml diff --git a/infra/charts/feast/charts/feast-core/requirements.yaml b/infra/charts/feast/charts/feast-core/requirements.yaml index efe9fec508a..ef1e39a7d0f 100644 --- a/infra/charts/feast/charts/feast-core/requirements.yaml +++ b/infra/charts/feast/charts/feast-core/requirements.yaml @@ -6,4 +6,10 @@ dependencies: - name: kafka version: 0.20.1 repository: "@incubator" - condition: kafka.enabled \ No newline at end of file + condition: kafka.enabled +- name: common + version: 0.0.5 + repository: "@incubator" +- name: prometheus-statsd-exporter + version: 0.1.2 + condition: prometheus-statsd-exporter.enabled \ No newline at end of file diff --git a/infra/charts/feast/charts/feast-core/templates/_ingress.yaml b/infra/charts/feast/charts/feast-core/templates/_ingress.yaml new file mode 100644 index 00000000000..5bed6df0470 --- /dev/null +++ b/infra/charts/feast/charts/feast-core/templates/_ingress.yaml @@ -0,0 +1,68 @@ +{{- /* +This takes an array of three values: +- the top context +- the feast component +- the service protocol +- the ingress context +*/ -}} +{{- define "feast.ingress" -}} +{{- $top := (index . 0) -}} +{{- $component := (index . 1) -}} +{{- $protocol := (index . 2) -}} +{{- $ingressValues := (index . 3) -}} +apiVersion: extensions/v1beta1 +kind: Ingress +{{ include "feast.ingress.metadata" . }} +spec: + rules: + {{- range $host := $ingressValues.hosts }} + - host: {{ $host }} + http: + paths: + - path: / + backend: + serviceName: {{ include (printf "feast-%s.fullname" $component) $top }} + servicePort: {{ index $top.Values "service" $protocol "port" }} + {{- end }} +{{- if $ingressValues.https.enabled }} + tls: + {{- range $host := $ingressValues.hosts }} + - secretName: {{ index $ingressValues.https.secretNames $host | default (splitList "." $host | rest | join "-" | printf "%s-tls") }} + hosts: + - {{ $host }} + {{- end }} +{{- end -}} +{{- end -}} + +{{- define "feast.ingress.metadata" -}} +{{- $commonMetadata := fromYaml (include "common.metadata" (first .)) }} +{{- $overrides := fromYaml (include "feast.ingress.metadata-overrides" .) -}} +{{- toYaml (merge $overrides $commonMetadata) -}} +{{- end -}} + +{{- define "feast.ingress.metadata-overrides" -}} +{{- $top := (index . 0) -}} +{{- $component := (index . 1) -}} +{{- $protocol := (index . 2) -}} +{{- $ingressValues := (index . 3) -}} +{{- $commonFullname := include "common.fullname" $top }} +metadata: + name: {{ $commonFullname }}-{{ $component }}-{{ $protocol }} + annotations: + kubernetes.io/ingress.class: {{ $ingressValues.class | quote }} + {{- if (and (eq $ingressValues.class "nginx") $ingressValues.auth.enabled) }} + nginx.ingress.kubernetes.io/auth-url: {{ $ingressValues.auth.authUrl | quote }} + nginx.ingress.kubernetes.io/auth-response-headers: "x-auth-request-email, x-auth-request-user" + nginx.ingress.kubernetes.io/auth-signin: "https://{{ $ingressValues.auth.signinHost | default (splitList "." (index $ingressValues.hosts 0) | rest | join "." | printf "auth.%s")}}/oauth2/start?rd=/r/$host/$request_uri" + {{- end }} + {{- if (and (eq $ingressValues.class "nginx") $ingressValues.whitelist) }} + nginx.ingress.kubernetes.io/whitelist-source-range: {{ $ingressValues.whitelist | quote -}} + {{- end }} + {{- if (and (eq $ingressValues.class "nginx") (eq $protocol "grpc") ) }} + # TODO: Allow choice of GRPC/GRPCS + nginx.ingress.kubernetes.io/backend-protocol: "GRPC" + {{- end }} + {{- if $ingressValues.annotations -}} + {{ include "common.annote" $ingressValues.annotations | indent 4 }} + {{- end }} +{{- end -}} diff --git a/infra/charts/feast/charts/feast-core/templates/configmap.yaml b/infra/charts/feast/charts/feast-core/templates/configmap.yaml index 68dc45c0571..da45cad5bdf 100644 --- a/infra/charts/feast/charts/feast-core/templates/configmap.yaml +++ b/infra/charts/feast/charts/feast-core/templates/configmap.yaml @@ -11,22 +11,43 @@ metadata: heritage: {{ .Release.Service }} data: application.yaml: | -{{- $config := index .Values "application.yaml"}} +{{- toYaml (index .Values "application.yaml") | nindent 4 }} {{- if .Values.postgresql.enabled }} -{{- $datasource := dict "url" (printf "jdbc:postgresql://%s:%s/%s" (printf "%s-postgresql" .Release.Name) (.Values.postgresql.service.port | toString) (.Values.postgresql.postgresqlDatabase)) "driverClassName" "org.postgresql.Driver" }} -{{- $newConfig := dict "spring" (dict "datasource" $datasource) }} -{{- $config := mergeOverwrite $config $newConfig }} + application-bundled-postgresql.yaml: | + spring: + datasource: + url: {{ printf "jdbc:postgresql://%s:%s/%s" (printf "%s-postgresql" .Release.Name) (.Values.postgresql.service.port | toString) (.Values.postgresql.postgresqlDatabase) }} + driverClassName: org.postgresql.Driver {{- end }} -{{- if .Values.kafka.enabled }} -{{- $topic := index .Values.kafka.topics 0 }} -{{- $options := dict "topic" $topic.name "replicationFactor" $topic.replicationFactor "partitions" $topic.partitions }} -{{- if not .Values.kafka.external.enabled }} -{{- $_ := set $options "bootstrapServers" (printf "%s:9092" (printf "%s-kafka" .Release.Name)) }} +{{ if .Values.kafka.enabled }} + {{- $topic := index .Values.kafka.topics 0 }} + application-bundled-kafka.yaml: | + feast: + stream: + type: kafka + options: + topic: {{ $topic.name | quote }} + replicationFactor: {{ $topic.replicationFactor }} + partitions: {{ $topic.partitions }} + {{- if not .Values.kafka.external.enabled }} + bootstrapServers: {{ printf "%s:9092" (printf "%s-kafka" .Release.Name) }} + {{- end }} {{- end }} -{{- $newConfig := dict "feast" (dict "stream" (dict "type" "kafka" "options" $options))}} -{{- $config := mergeOverwrite $config $newConfig }} + +{{- if (index .Values "prometheus-statsd-exporter" "enabled" )}} + application-bundled-statsd.yaml: | + feast: + jobs: + metrics: + enabled: true + type: statsd + host: prometheus-statsd-exporter + port: 9125 {{- end }} -{{- toYaml $config | nindent 4 }} +{{- range $name, $content := .Values.springConfigProfiles }} + application-{{ $name }}.yaml: | +{{- toYaml $content | nindent 4 }} +{{- end }} diff --git a/infra/charts/feast/charts/feast-core/templates/deployment.yaml b/infra/charts/feast/charts/feast-core/templates/deployment.yaml index 0671d9574b3..df834b6749e 100644 --- a/infra/charts/feast/charts/feast-core/templates/deployment.yaml +++ b/infra/charts/feast/charts/feast-core/templates/deployment.yaml @@ -18,6 +18,13 @@ spec: release: {{ .Release.Name }} template: metadata: + {{- if .Values.prometheus.enabled }} + annotations: + {{ $config := index .Values "application.yaml" }} + prometheus.io/path: /metrics + prometheus.io/port: "{{ $config.server.port }}" + prometheus.io/scrape: "true" + {{- end }} labels: app: {{ template "feast-core.name" . }} component: core @@ -42,7 +49,7 @@ spec: - name: {{ .Chart.Name }} image: '{{ .Values.image.repository }}:{{ required "No .image.tag found. This must be provided as input." .Values.image.tag }}' imagePullPolicy: {{ .Values.image.pullPolicy }} - + volumeMounts: - name: {{ template "feast-core.fullname" . }}-config mountPath: "{{ .Values.springConfigMountPath }}" @@ -53,31 +60,48 @@ spec: {{- end }} env: + - name: LOG_TYPE + value: {{ .Values.logType | quote }} + - name: LOG_LEVEL + value: {{ .Values.logLevel | quote }} + {{- if .Values.postgresql.enabled }} - name: SPRING_DATASOURCE_USERNAME - value: {{ .Values.postgresql.postgresqlUsername }} + value: {{ .Values.postgresql.postgresqlUsername | quote }} - name: SPRING_DATASOURCE_PASSWORD - value: {{ .Values.postgresql.postgresqlPassword }} + value: {{ .Values.postgresql.postgresqlPassword | quote }} {{- end }} {{- if .Values.gcpServiceAccount.useExistingSecret }} - name: GOOGLE_APPLICATION_CREDENTIALS value: {{ .Values.gcpServiceAccount.mountPath }}/{{ .Values.gcpServiceAccount.existingSecret.key }} {{- end }} + {{- if .Values.gcpProjectId }} + - name: GOOGLE_CLOUD_PROJECT + value: {{ .Values.gcpProjectId | quote }} + {{- end }} command: - java {{- range .Values.jvmOptions }} - - {{ . }} + - {{ . | quote }} + {{- end }} + - -jar + - {{ .Values.jarPath | quote }} + - "--spring.config.location=file:{{ .Values.springConfigMountPath }}/" + {{- $profilesArray := splitList "," .Values.springConfigProfilesActive -}} + {{- $profilesArray = append $profilesArray (.Values.postgresql.enabled | ternary "bundled-postgresql" "") -}} + {{- $profilesArray = append $profilesArray (.Values.kafka.enabled | ternary "bundled-kafka" "") -}} + {{- $profilesArray = append $profilesArray (index .Values "prometheus-statsd-exporter" "enabled" | ternary "bundled-statsd" "") -}} + {{- $profilesArray = compact $profilesArray -}} + {{- if $profilesArray }} + - "--spring.profiles.active={{ join "," $profilesArray }}" {{- end }} - - -jar - - /opt/feast/feast-core.jar - - "--spring.config.location=file:{{ .Values.springConfigMountPath }}/application.yaml" ports: - name: http containerPort: {{ .Values.service.http.targetPort }} - - name: grpc + - name: grpc containerPort: {{ .Values.service.grpc.targetPort }} {{- if .Values.livenessProbe.enabled }} @@ -103,6 +127,6 @@ spec: timeoutSeconds: {{ .Values.readinessProbe.timeoutSeconds }} failureThreshold: {{ .Values.readinessProbe.failureThreshold }} {{- end }} - + resources: {{- toYaml .Values.resources | nindent 10 }} diff --git a/infra/charts/feast/charts/feast-core/templates/ingress.yaml b/infra/charts/feast/charts/feast-core/templates/ingress.yaml index 86fc2d3f175..7f453e1a75f 100644 --- a/infra/charts/feast/charts/feast-core/templates/ingress.yaml +++ b/infra/charts/feast/charts/feast-core/templates/ingress.yaml @@ -1,28 +1,7 @@ -{{- if .Values.ingress.enabled -}} -{{- $fullName := include "feast-core.fullname" . -}} -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: {{ $fullName }} - labels: - app: {{ template "feast-core.name" . }} - chart: {{ .Chart.Name }}-{{ .Chart.Version }} - component: core - heritage: {{ .Release.Service }} - release: {{ .Release.Name }} - annotations: -{{- with .Values.ingress.annotations }} -{{ toYaml . | indent 4 }} +{{- if .Values.ingress.http.enabled -}} +{{ template "feast.ingress" (list . "core" "http" .Values.ingress.http) }} +{{- end }} +--- +{{ if .Values.ingress.grpc.enabled -}} +{{ template "feast.ingress" (list . "core" "grpc" .Values.ingress.grpc) }} {{- end }} -spec: - rules: - {{- range .Values.ingress.hosts }} - - host: {{ .host | quote }} - http: - paths: - - path: / - backend: - serviceName: {{ $fullName }} - servicePort: {{ .port | quote }} - {{- end }} -{{- end }} \ No newline at end of file diff --git a/infra/charts/feast/charts/feast-core/values.yaml b/infra/charts/feast/charts/feast-core/values.yaml index f746bc96ead..077906dc35d 100644 --- a/infra/charts/feast/charts/feast-core/values.yaml +++ b/infra/charts/feast/charts/feast-core/values.yaml @@ -1,12 +1,15 @@ -# postgresql configures Postgresql that is installed as part of Feast Core. +# ============================================================ +# Bundled PostgreSQL +# ============================================================ + # Refer to https://github.com/helm/charts/tree/c42002a21abf8eff839ff1d2382152bde2bbe596/stable/postgresql # for additional configuration. postgresql: # enabled specifies whether Postgresql should be installed as part of Feast Core. # - # Feast Core requires a database to store data such as the created FeatureSets + # Feast Core requires a database to store data such as the created FeatureSets # and job statuses. If enabled, the database and service port specified below - # will override "spring.datasource.url" value in application.yaml. The + # will override "spring.datasource.url" value in application.yaml. The # username and password will also be set as environment variables that will # override "spring.datasource.username/password" in application.yaml. enabled: true @@ -20,12 +23,15 @@ postgresql: # port is the TCP port that Postgresql will listen to port: 5432 -# kafka configures Kafka that is installed as part of Feast Core. +# ============================================================ +# Bundled Kafka +# ============================================================ + # Refer to https://github.com/helm/charts/tree/c42002a21abf8eff839ff1d2382152bde2bbe596/incubator/kafka # for additional configuration. kafka: # enabled specifies whether Kafka should be installed as part of Feast Core. - # + # # Feast Core requires a Kafka instance to be set as the default source for # FeatureRows. If enabled, "feast.stream" option in application.yaml will # be overridden by this installed Kafka configuration. @@ -36,6 +42,18 @@ kafka: replicationFactor: 1 partitions: 1 + +# ============================================================ +# Bundled Prometheus StatsD Exporter +# ============================================================ + +prometheus-statsd-exporter: + enabled: false + +# ============================================================ +# Feast Core +# ============================================================ + # replicaCount is the number of pods that will be created. replicaCount: 1 @@ -44,13 +62,18 @@ image: repository: gcr.io/kf-feast/feast-core pullPolicy: IfNotPresent +# Add prometheus scraping annotations to the Pod metadata. +# If enabled, you must also ensure server.port is specified under application.yaml +prometheus: + enabled: false + # application.yaml is the main configuration for Feast Core application. -# +# # Feast Core is a Spring Boot app which uses this yaml configuration file. # Refer to https://github.com/gojek/feast/blob/79eb4ab5fa3d37102c1dca9968162a98690526ba/core/src/main/resources/application.yml # for a complete list and description of the configuration. # -# Note that some properties defined in application.yaml may be overriden by +# Note that some properties defined in application.yaml may be overriden by # Helm under certain conditions. For example, if postgresql and kafka dependencies # are enabled. application.yaml: @@ -96,7 +119,14 @@ application.yaml: host: localhost port: 8125 -# springConfigMountPath is the directory path where application.yaml will be +springConfigProfiles: {} +# db: | +# spring: +# datasource: +# driverClassName: org.postgresql.Driver +# url: jdbc:postgresql://${DB_HOST:127.0.0.1}:${DB_PORT:5432}/${DB_DATABASE:postgres} +springConfigProfilesActive: "" +# springConfigMountPath is the directory path where application.yaml will be # mounted in the container. springConfigMountPath: /etc/feast/feast-core @@ -107,7 +137,7 @@ gcpServiceAccount: useExistingSecret: false existingSecret: # name is the secret name of the existing secret for the service account. - name: feast-gcp-service-account + name: feast-gcp-service-account # key is the secret key of the existing secret for the service account. # key is normally derived from the file name of the JSON key file. key: key.json @@ -115,19 +145,29 @@ gcpServiceAccount: # the value of "existingSecret.key" is file name of the service account file. mountPath: /etc/gcloud/service-accounts -# jvmOptions are options that will be passed to the Java Virtual Machine (JVM) +# Project ID picked up by the Cloud SDK (e.g. BigQuery run against this project) +gcpProjectId: "" + +# Path to Jar file in the Docker image. +# If you are using gcr.io/kf-feast/feast-core this should not need to be changed +jarPath: /opt/feast/feast-core.jar + +# jvmOptions are options that will be passed to the Java Virtual Machine (JVM) # running Feast Core. -# +# # For example, it is good practice to set min and max heap size in JVM. # https://stackoverflow.com/questions/6902135/side-effect-for-increasing-maxpermsize-and-max-heap-size # # Refer to https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html # to see other JVM options that can be set. # -# jvmOptions: -# - -Xms1024m +jvmOptions: [] +# - -Xms1024m # - -Xmx1024m +logType: JSON +logLevel: warn + livenessProbe: enabled: true initialDelaySeconds: 60 @@ -162,12 +202,29 @@ service: # nodePort: ingress: - enabled: false - annotations: {} - # kubernetes.io/ingress.class: nginx - hosts: - # - host: chart-example.local - # port: http + grpc: + enabled: false + class: nginx + hosts: [] + annotations: {} + https: + enabled: true + secretNames: {} + whitelist: "" + auth: + enabled: false + http: + enabled: false + class: nginx + hosts: [] + annotations: {} + https: + enabled: true + secretNames: {} + whitelist: "" + auth: + enabled: false + authUrl: http://auth-server.auth-ns.svc.cluster.local/auth resources: {} # We usually recommend not to specify default resources and to leave this as a conscious diff --git a/infra/charts/feast/charts/feast-serving/requirements.yaml b/infra/charts/feast/charts/feast-serving/requirements.yaml index fa4c1df4c10..2cee3f81494 100644 --- a/infra/charts/feast/charts/feast-serving/requirements.yaml +++ b/infra/charts/feast/charts/feast-serving/requirements.yaml @@ -3,3 +3,6 @@ dependencies: version: 9.5.0 repository: "@stable" condition: redis.enabled +- name: common + version: 0.0.5 + repository: "@incubator" diff --git a/infra/charts/feast/charts/feast-serving/templates/_helpers.tpl b/infra/charts/feast/charts/feast-serving/templates/_helpers.tpl index 49abb6b8e50..ab670cc8cc7 100644 --- a/infra/charts/feast/charts/feast-serving/templates/_helpers.tpl +++ b/infra/charts/feast/charts/feast-serving/templates/_helpers.tpl @@ -43,3 +43,10 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} {{- end }} app.kubernetes.io/managed-by: {{ .Release.Service }} {{- end -}} + +{{/* +Helpers +*/}} +{{- define "bq_store_and_no_job_options" -}} +{{ and (eq (index .Values "store.yaml" "type") "BIGQUERY") (empty (index .Values "application.yaml" "feast" "jobs" "store-options")) }} +{{- end -}} diff --git a/infra/charts/feast/charts/feast-serving/templates/_ingress.yaml b/infra/charts/feast/charts/feast-serving/templates/_ingress.yaml new file mode 100644 index 00000000000..5bed6df0470 --- /dev/null +++ b/infra/charts/feast/charts/feast-serving/templates/_ingress.yaml @@ -0,0 +1,68 @@ +{{- /* +This takes an array of three values: +- the top context +- the feast component +- the service protocol +- the ingress context +*/ -}} +{{- define "feast.ingress" -}} +{{- $top := (index . 0) -}} +{{- $component := (index . 1) -}} +{{- $protocol := (index . 2) -}} +{{- $ingressValues := (index . 3) -}} +apiVersion: extensions/v1beta1 +kind: Ingress +{{ include "feast.ingress.metadata" . }} +spec: + rules: + {{- range $host := $ingressValues.hosts }} + - host: {{ $host }} + http: + paths: + - path: / + backend: + serviceName: {{ include (printf "feast-%s.fullname" $component) $top }} + servicePort: {{ index $top.Values "service" $protocol "port" }} + {{- end }} +{{- if $ingressValues.https.enabled }} + tls: + {{- range $host := $ingressValues.hosts }} + - secretName: {{ index $ingressValues.https.secretNames $host | default (splitList "." $host | rest | join "-" | printf "%s-tls") }} + hosts: + - {{ $host }} + {{- end }} +{{- end -}} +{{- end -}} + +{{- define "feast.ingress.metadata" -}} +{{- $commonMetadata := fromYaml (include "common.metadata" (first .)) }} +{{- $overrides := fromYaml (include "feast.ingress.metadata-overrides" .) -}} +{{- toYaml (merge $overrides $commonMetadata) -}} +{{- end -}} + +{{- define "feast.ingress.metadata-overrides" -}} +{{- $top := (index . 0) -}} +{{- $component := (index . 1) -}} +{{- $protocol := (index . 2) -}} +{{- $ingressValues := (index . 3) -}} +{{- $commonFullname := include "common.fullname" $top }} +metadata: + name: {{ $commonFullname }}-{{ $component }}-{{ $protocol }} + annotations: + kubernetes.io/ingress.class: {{ $ingressValues.class | quote }} + {{- if (and (eq $ingressValues.class "nginx") $ingressValues.auth.enabled) }} + nginx.ingress.kubernetes.io/auth-url: {{ $ingressValues.auth.authUrl | quote }} + nginx.ingress.kubernetes.io/auth-response-headers: "x-auth-request-email, x-auth-request-user" + nginx.ingress.kubernetes.io/auth-signin: "https://{{ $ingressValues.auth.signinHost | default (splitList "." (index $ingressValues.hosts 0) | rest | join "." | printf "auth.%s")}}/oauth2/start?rd=/r/$host/$request_uri" + {{- end }} + {{- if (and (eq $ingressValues.class "nginx") $ingressValues.whitelist) }} + nginx.ingress.kubernetes.io/whitelist-source-range: {{ $ingressValues.whitelist | quote -}} + {{- end }} + {{- if (and (eq $ingressValues.class "nginx") (eq $protocol "grpc") ) }} + # TODO: Allow choice of GRPC/GRPCS + nginx.ingress.kubernetes.io/backend-protocol: "GRPC" + {{- end }} + {{- if $ingressValues.annotations -}} + {{ include "common.annote" $ingressValues.annotations | indent 4 }} + {{- end }} +{{- end -}} diff --git a/infra/charts/feast/charts/feast-serving/templates/configmap.yaml b/infra/charts/feast/charts/feast-serving/templates/configmap.yaml index 0ec80252c16..934216a9d5f 100644 --- a/infra/charts/feast/charts/feast-serving/templates/configmap.yaml +++ b/infra/charts/feast/charts/feast-serving/templates/configmap.yaml @@ -11,37 +11,43 @@ metadata: heritage: {{ .Release.Service }} data: application.yaml: | -{{- $config := index .Values "application.yaml" }} +{{- toYaml (index .Values "application.yaml") | nindent 4 }} {{- if .Values.core.enabled }} -{{- $newConfig := dict "feast" (dict "core-host" (printf "%s-feast-core" .Release.Name)) }} -{{- $config := mergeOverwrite $config $newConfig }} + application-bundled-core.yaml: | + feast: + core-host: {{ printf "%s-feast-core" .Release.Name }} {{- end }} -{{- $store := index .Values "store.yaml" }} -{{- if and (eq $store.type "BIGQUERY") (not (hasKey $config.feast.jobs "store-options")) }} -{{- $jobStore := dict "host" (printf "%s-redis-headless" .Release.Name) "port" 6379 }} -{{- $newConfig := dict "feast" (dict "jobs" (dict "store-options" $jobStore)) }} -{{- $config := mergeOverwrite $config $newConfig }} +{{- if eq (include "bq_store_and_no_job_options" .) "true" }} + application-bundled-redis.yaml: | + feast: + jobs: + store-options: + host: {{ printf "%s-redis-headless" .Release.Name }} + port: 6379 {{- end }} -{{- toYaml $config | nindent 4 }} - store.yaml: | -{{- $config := index .Values "store.yaml"}} +{{- $store := index .Values "store.yaml"}} -{{- if and .Values.redis.enabled (eq $config.type "REDIS") }} +{{- if and .Values.redis.enabled (eq $store.type "REDIS") }} {{- if eq .Values.redis.master.service.type "ClusterIP" }} {{- $newConfig := dict "redis_config" (dict "host" (printf "%s-redis-headless" .Release.Name) "port" .Values.redis.redisPort) }} -{{- $config := mergeOverwrite $config $newConfig }} +{{- $config := mergeOverwrite $store $newConfig }} {{- end }} {{- if and (eq .Values.redis.master.service.type "LoadBalancer") (not (empty .Values.redis.master.service.loadBalancerIP)) }} {{- $newConfig := dict "redis_config" (dict "host" .Values.redis.master.service.loadBalancerIP "port" .Values.redis.redisPort) }} -{{- $config := mergeOverwrite $config $newConfig }} +{{- $config := mergeOverwrite $store $newConfig }} {{- end }} {{- end }} -{{- toYaml $config | nindent 4 }} +{{- toYaml $store | nindent 4 }} + +{{- range $name, $content := .Values.springConfigProfiles }} + application-{{ $name }}.yaml: | +{{- toYaml $content | nindent 4 }} +{{- end }} diff --git a/infra/charts/feast/charts/feast-serving/templates/deployment.yaml b/infra/charts/feast/charts/feast-serving/templates/deployment.yaml index e6824a23465..64dd3955d0c 100644 --- a/infra/charts/feast/charts/feast-serving/templates/deployment.yaml +++ b/infra/charts/feast/charts/feast-serving/templates/deployment.yaml @@ -49,7 +49,7 @@ spec: - name: {{ .Chart.Name }} image: '{{ .Values.image.repository }}:{{ required "No .image.tag found. This must be provided as input." .Values.image.tag }}' imagePullPolicy: {{ .Values.image.pullPolicy }} - + volumeMounts: - name: {{ template "feast-serving.fullname" . }}-config mountPath: "{{ .Values.springConfigMountPath }}" @@ -60,24 +60,40 @@ spec: {{- end }} env: + - name: LOG_TYPE + value: {{ .Values.logType | quote }} + - name: LOG_LEVEL + value: {{ .Values.logLevel | quote }} + {{- if .Values.gcpServiceAccount.useExistingSecret }} - name: GOOGLE_APPLICATION_CREDENTIALS value: {{ .Values.gcpServiceAccount.mountPath }}/{{ .Values.gcpServiceAccount.existingSecret.key }} {{- end }} + {{- if .Values.gcpProjectId }} + - name: GOOGLE_CLOUD_PROJECT + value: {{ .Values.gcpProjectId | quote }} + {{- end }} command: - java {{- range .Values.jvmOptions }} - - {{ . }} + - {{ . | quote }} + {{- end }} + - -jar + - {{ .Values.jarPath | quote }} + - "--spring.config.location=file:{{ .Values.springConfigMountPath }}/" + {{- $profilesArray := splitList "," .Values.springConfigProfilesActive -}} + {{- $profilesArray = append $profilesArray (.Values.core.enabled | ternary "bundled-core" "") -}} + {{- $profilesArray = append $profilesArray (eq (include "bq_store_and_no_job_options" .) "true" | ternary "bundled-redis" "") -}} + {{- $profilesArray = compact $profilesArray -}} + {{- if $profilesArray }} + - "--spring.profiles.active={{ join "," $profilesArray }}" {{- end }} - - -jar - - /opt/feast/feast-serving.jar - - "--spring.config.location=file:{{ .Values.springConfigMountPath }}/application.yaml" ports: - name: http containerPort: {{ .Values.service.http.targetPort }} - - name: grpc + - name: grpc containerPort: {{ .Values.service.grpc.targetPort }} {{- if .Values.livenessProbe.enabled }} @@ -101,6 +117,6 @@ spec: timeoutSeconds: {{ .Values.readinessProbe.timeoutSeconds }} failureThreshold: {{ .Values.readinessProbe.failureThreshold }} {{- end }} - + resources: {{- toYaml .Values.resources | nindent 10 }} diff --git a/infra/charts/feast/charts/feast-serving/templates/ingress.yaml b/infra/charts/feast/charts/feast-serving/templates/ingress.yaml index c6b4cb07a81..1bcd176147a 100644 --- a/infra/charts/feast/charts/feast-serving/templates/ingress.yaml +++ b/infra/charts/feast/charts/feast-serving/templates/ingress.yaml @@ -1,28 +1,7 @@ -{{- if .Values.ingress.enabled -}} -{{- $fullName := include "feast-serving.fullname" . -}} -apiVersion: extensions/v1beta1 -kind: Ingress -metadata: - name: {{ $fullName }} - labels: - app: {{ template "feast-serving.name" . }} - chart: {{ .Chart.Name }}-{{ .Chart.Version }} - component: serving - heritage: {{ .Release.Service }} - release: {{ .Release.Name }} - annotations: -{{- with .Values.ingress.annotations }} -{{ toYaml . | indent 4 }} +{{- if .Values.ingress.http.enabled -}} +{{ template "feast.ingress" (list . "serving" "http" .Values.ingress.http) }} {{- end }} -spec: - rules: - {{- range .Values.ingress.hosts }} - - host: {{ .host | quote }} - http: - paths: - - path: / - backend: - serviceName: {{ $fullName }} - servicePort: {{ .port | quote }} - {{- end }} +--- +{{ if .Values.ingress.grpc.enabled -}} +{{ template "feast.ingress" (list . "serving" "grpc" .Values.ingress.grpc) }} {{- end }} diff --git a/infra/charts/feast/charts/feast-serving/values.yaml b/infra/charts/feast/charts/feast-serving/values.yaml index d2b3c599479..52d10cd7440 100644 --- a/infra/charts/feast/charts/feast-serving/values.yaml +++ b/infra/charts/feast/charts/feast-serving/values.yaml @@ -3,23 +3,23 @@ # for additional configuration redis: # enabled specifies whether Redis should be installed as part of Feast Serving. - # + # # If enabled, "redis_config" in store.yaml will be overwritten by Helm # to the configuration in this Redis installation. enabled: false # usePassword specifies if password is required to access Redis. Note that # Feast 0.3 does not support Redis with password. - usePassword: false + usePassword: false # cluster configuration for Redis. cluster: # enabled specifies if Redis should be installed in cluster mode. enabled: false -# core configures Feast Core in the same parent feast chart that this Feast +# core configures Feast Core in the same parent feast chart that this Feast # Serving connects to. core: # enabled specifies that Feast Serving will use Feast Core installed - # in the same parent feast chart. If enabled, Helm will overwrite + # in the same parent feast chart. If enabled, Helm will overwrite # "feast.core-host" in application.yaml with the correct value. enabled: true @@ -37,7 +37,7 @@ image: # Refer to https://github.com/gojek/feast/blob/79eb4ab5fa3d37102c1dca9968162a98690526ba/serving/src/main/resources/application.yml # for a complete list and description of the configuration. # -# Note that some properties defined in application.yaml may be overridden by +# Note that some properties defined in application.yaml may be overridden by # Helm under certain conditions. For example, if core is enabled, then # "feast.core-host" will be overridden. Also, if "type: BIGQUERY" is specified # in store.yaml, "feast.jobs.store-options" will be overridden as well with @@ -66,19 +66,19 @@ application.yaml: port: 8080 # store.yaml is the configuration for Feast Store. -# +# # Refer to this link for description: # https://github.com/gojek/feast/blob/79eb4ab5fa3d37102c1dca9968162a98690526ba/protos/feast/core/Store.proto # # Use the correct store configuration depending on whether the installed # Feast Serving is "online" or "batch", by uncommenting the correct store.yaml. # -# Note that if "redis.enabled: true" and "type: REDIS" in store.yaml, +# Note that if "redis.enabled: true" and "type: REDIS" in store.yaml, # Helm will override "redis_config" with configuration of Redis installed # in this chart. -# +# # Note that if "type: BIGQUERY" in store.yaml, Helm assumes Feast Online serving -# is also installed with Redis store. Helm will then override "feast.jobs.store-options" +# is also installed with Redis store. Helm will then override "feast.jobs.store-options" # in application.yaml with the installed Redis store configuration. This is # because in Feast 0.3, Redis job store is required. # @@ -104,7 +104,14 @@ application.yaml: # name: "*" # version: "*" -# springConfigMountPath is the directory path where application.yaml and +springConfigProfiles: {} +# db: | +# spring: +# datasource: +# driverClassName: org.postgresql.Driver +# url: jdbc:postgresql://${DB_HOST:127.0.0.1}:${DB_PORT:5432}/${DB_DATABASE:postgres} +springConfigProfilesActive: "" +# springConfigMountPath is the directory path where application.yaml and # store.yaml will be mounted in the container. springConfigMountPath: /etc/feast/feast-serving @@ -115,7 +122,7 @@ gcpServiceAccount: useExistingSecret: false existingSecret: # name is the secret name of the existing secret for the service account. - name: feast-gcp-service-account + name: feast-gcp-service-account # key is the secret key of the existing secret for the service account. # key is normally derived from the file name of the JSON key file. key: key.json @@ -123,19 +130,29 @@ gcpServiceAccount: # the value of "existingSecret.key" is file name of the service account file. mountPath: /etc/gcloud/service-accounts -# jvmOptions are options that will be passed to the Java Virtual Machine (JVM) +# Project ID picked up by the Cloud SDK (e.g. BigQuery run against this project) +gcpProjectId: "" + +# Path to Jar file in the Docker image. +# If using gcr.io/kf-feast/feast-serving this should not need to be changed. +jarPath: /opt/feast/feast-serving.jar + +# jvmOptions are options that will be passed to the Java Virtual Machine (JVM) # running Feast Core. -# +# # For example, it is good practice to set min and max heap size in JVM. # https://stackoverflow.com/questions/6902135/side-effect-for-increasing-maxpermsize-and-max-heap-size # # Refer to https://docs.oracle.com/cd/E22289_01/html/821-1274/configuring-the-default-jvm-and-java-arguments.html # to see other JVM options that can be set. # -# jvmOptions: -# - -Xms768m +jvmOptions: [] +# - -Xms768m # - -Xmx768m +logType: JSON +logLevel: warn + livenessProbe: enabled: false initialDelaySeconds: 60 @@ -170,12 +187,29 @@ service: # nodePort: ingress: - enabled: false - annotations: {} - # kubernetes.io/ingress.class: nginx - hosts: - # - host: chart-example.local - # port: http + grpc: + enabled: false + class: nginx + hosts: [] + annotations: {} + https: + enabled: true + secretNames: {} + whitelist: "" + auth: + enabled: false + http: + enabled: false + class: nginx + hosts: [] + annotations: {} + https: + enabled: true + secretNames: {} + whitelist: "" + auth: + enabled: false + authUrl: http://auth-server.auth-ns.svc.cluster.local/auth prometheus: enabled: true @@ -185,6 +219,7 @@ resources: {} # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. + # # limits: # cpu: 100m # memory: 128Mi diff --git a/infra/charts/feast/requirements.lock b/infra/charts/feast/requirements.lock index 8afd9521573..e441790dc76 100644 --- a/infra/charts/feast/requirements.lock +++ b/infra/charts/feast/requirements.lock @@ -1,12 +1,6 @@ dependencies: -- name: feast-core - repository: "" - version: 0.3.2 -- name: feast-serving - repository: "" - version: 0.3.2 -- name: feast-serving - repository: "" - version: 0.3.2 -digest: sha256:7ee4cd271cbd4ace44817dd12ba65f490a8e3529adf199604a2c2bdad9c2fac3 -generated: "2019-11-27T13:35:41.334054+08:00" +- name: common + repository: https://kubernetes-charts-incubator.storage.googleapis.com + version: 0.0.5 +digest: sha256:935bfb09e9ed90ff800826a7df21adaabe3225511c3ad78df44e1a5a60e93f14 +generated: 2019-12-10T14:47:49.57569Z diff --git a/infra/charts/feast/requirements.yaml b/infra/charts/feast/requirements.yaml index 5416ded3fee..1fa1826965a 100644 --- a/infra/charts/feast/requirements.yaml +++ b/infra/charts/feast/requirements.yaml @@ -9,4 +9,4 @@ dependencies: - name: feast-serving alias: feast-serving-online version: 0.4.4 - condition: feast-serving-online.enabled + condition: feast-serving-online.enabled \ No newline at end of file diff --git a/infra/charts/feast/values-demo.yaml b/infra/charts/feast/values-demo.yaml index fad4bc0afb0..2cb5ccbe741 100644 --- a/infra/charts/feast/values-demo.yaml +++ b/infra/charts/feast/values-demo.yaml @@ -1,7 +1,7 @@ # The following are values for installing Feast for demonstration purpose: # - Persistence is disabled since for demo purpose data is not expected # to be durable -# - Only online serving (no batch serving) is installed to remove dependency +# - Only online serving (no batch serving) is installed to remove dependency # on Google Cloud services. Batch serving requires BigQuery dependency. # - Replace all occurrences of "feast.example.com" with the domain name or # external IP pointing to your cluster @@ -68,4 +68,17 @@ feast-serving-online: version: "*" feast-serving-batch: - enabled: false +# enabled: false + enabled: true + store.yaml: + name: bigquery + type: BIGQUERY + bigquery_config: + project_id: PROJECT_ID + dataset_id: DATASET_ID + subscriptions: + - project: "*" + name: "*" + version: "*" + redis: + enabled: false \ No newline at end of file diff --git a/infra/charts/feast/values.yaml b/infra/charts/feast/values.yaml index f9a0a76dc1b..fde03f9ad71 100644 --- a/infra/charts/feast/values.yaml +++ b/infra/charts/feast/values.yaml @@ -2,10 +2,12 @@ # - Feast Core # - Feast Serving Online # - Feast Serving Batch +# - Prometheus StatsD Exporter # # The configuration for different components can be referenced from: # - charts/feast-core/values.yaml # - charts/feast-serving/values.yaml +# - charts/prometheus-statsd-exporter/values.yaml # # Note that "feast-serving-online" and "feast-serving-batch" are # aliases to "feast-serving" chart since in typical scenario two instances @@ -235,11 +237,11 @@ feast-serving-batch: # enabled as well. So Feast Serving Batch will share the same # Redis instance to store job statuses. store-type: REDIS - store-options: - # Use the externally exposed redis instance deployed by Online service - # Please set EXTERNAL_IP to your cluster's external IP - host: EXTERNAL_IP - port: 32101 + # Default to use the internal hostname of the redis instance deployed by Online service, + # otherwise use externally exposed by setting EXTERNAL_IP to your cluster's external IP + # store-options: + # host: EXTERNAL_IP + # port: 32101 # store.yaml is the configuration for Feast Store. # # Refer to this link for more description: