Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
[storage] Add an SQL storage driver
This commits adds the possibility to back Tiller (or the future
Tiller-less Helm CLI) with any SQL database (only postgres has been
tested so far) to store release information.

The main motivation for this commit was to use a storage backend that
would allow releases larger that 1MB in size (ConfigMap or Secret
drivers don't, because of limits on value size in the underlying etcd
key-value store).

Signed-off-by: Étienne Lafarge <etienne.lafarge@gmail.com>

Co-authored-by: Elliot Maincourt <e.maincourt@gmail.com> (@emaincourt)
Co-authored-by: Paul Borensztein <hi@0x01.fr> (@commit-master)
  • Loading branch information
elafarge committed Feb 27, 2019
1 parent 42e424b commit 531aaaa
Show file tree
Hide file tree
Showing 8 changed files with 911 additions and 88 deletions.
38 changes: 28 additions & 10 deletions cmd/tiller/tiller.go
Expand Up @@ -65,6 +65,7 @@ const (
storageMemory = "memory"
storageConfigMap = "configmap"
storageSecret = "secret"
storageSQL = "sql"

probeAddr = ":44135"
traceAddr = ":44136"
Expand All @@ -74,17 +75,22 @@ const (
)

var (
grpcAddr = flag.String("listen", ":44134", "address:port to listen on")
enableTracing = flag.Bool("trace", false, "enable rpc tracing")
store = flag.String("storage", storageConfigMap, "storage driver to use. One of 'configmap', 'memory', or 'secret'")
grpcAddr = flag.String("listen", ":44134", "address:port to listen on")
enableTracing = flag.Bool("trace", false, "enable rpc tracing")

store = flag.String("storage", storageConfigMap, "storage driver to use. One of 'configmap', 'memory', 'sql' or 'secret'")
sqlDialect = flag.String("sql-dialect", "postgres", "SQL dialect to use (only postgres is supported for now")
sqlConnectionString = flag.String("sql-connection-string", "", "SQL connection string to use")

remoteReleaseModules = flag.Bool("experimental-release", false, "enable experimental release modules")
tlsEnable = flag.Bool("tls", tlsEnableEnvVarDefault(), "enable TLS")
tlsVerify = flag.Bool("tls-verify", tlsVerifyEnvVarDefault(), "enable TLS and verify remote certificate")
keyFile = flag.String("tls-key", tlsDefaultsFromEnv("tls-key"), "path to TLS private key file")
certFile = flag.String("tls-cert", tlsDefaultsFromEnv("tls-cert"), "path to TLS certificate file")
caCertFile = flag.String("tls-ca-cert", tlsDefaultsFromEnv("tls-ca-cert"), "trust certificates signed by this CA")
maxHistory = flag.Int("history-max", historyMaxFromEnv(), "maximum number of releases kept in release history, with 0 meaning no limit")
printVersion = flag.Bool("version", false, "print the version number")

tlsEnable = flag.Bool("tls", tlsEnableEnvVarDefault(), "enable TLS")
tlsVerify = flag.Bool("tls-verify", tlsVerifyEnvVarDefault(), "enable TLS and verify remote certificate")
keyFile = flag.String("tls-key", tlsDefaultsFromEnv("tls-key"), "path to TLS private key file")
certFile = flag.String("tls-cert", tlsDefaultsFromEnv("tls-cert"), "path to TLS certificate file")
caCertFile = flag.String("tls-ca-cert", tlsDefaultsFromEnv("tls-ca-cert"), "trust certificates signed by this CA")
maxHistory = flag.Int("history-max", historyMaxFromEnv(), "maximum number of releases kept in release history, with 0 meaning no limit")
printVersion = flag.Bool("version", false, "print the version number")

// rootServer is the root gRPC server.
//
Expand Down Expand Up @@ -141,6 +147,18 @@ func start() {

env.Releases = storage.Init(secrets)
env.Releases.Log = newLogger("storage").Printf
case storageSQL:
sqlDriver, err := driver.NewSQL(
*sqlDialect,
*sqlConnectionString,
newLogger("storage/driver").Printf,
)
if err != nil {
logger.Fatalf("Cannot initialize SQL storage driver: %v", err)
}

env.Releases = storage.Init(sqlDriver)
env.Releases.Log = newLogger("storage").Printf
}

if *maxHistory > 0 {
Expand Down
34 changes: 31 additions & 3 deletions docs/install.md
Expand Up @@ -353,10 +353,13 @@ in JSON format.

### Storage backends
By default, `tiller` stores release information in `ConfigMaps` in the namespace
where it is running. As of Helm 2.7.0, there is now a beta storage backend that
where it is running.

#### Secret storage backend
As of Helm 2.7.0, there is now a beta storage backend that
uses `Secrets` for storing release information. This was added for additional
security in protecting charts in conjunction with the release of `Secret`
encryption in Kubernetes.
security in protecting charts in conjunction with the release of `Secret`
encryption in Kubernetes.

To enable the secrets backend, you'll need to init Tiller with the following
options:
Expand All @@ -369,6 +372,31 @@ Currently, if you want to switch from the default backend to the secrets
backend, you'll have to do the migration for this on your own. When this backend
graduates from beta, there will be a more official path of migration

#### SQL storage backend
As of Helm 2.14.0 there is now a beta SQL storage backend that stores release
information in an SQL database (only postgres has been tested so far).

Using such a storage backend is particularly useful if your release information
weighs more than 1MB (in which case, it can't be stored in ConfigMaps/Secrets
because of internal limits in Kubernetes' underlying etcd key-value store).

To enable the SQL backend, you'll need to [deploy an SQL
database](./sql-storage.md) and init Tiller with the following options:

```shell
helm init \
--override \
'spec.template.spec.containers[0].args'='{--storage=sql,--sql-dialect=postgres,--sql-connection-string=postgresql://tiller-postgres:5432/helm?user=helm&password=changemeforgodssake&sslmode=disable}'
```

**PRODUCTION NOTES**: it's recommended to change the username and password of
the SQL database in production deployments. Enabling SSL is also a good idea.
Last, but not least, perform regular backups/snapshots of your SQL database.

Currently, if you want to switch from the default backend to the SQL backend,
you'll have to do the migration for this on your own. When this backend
graduates from beta, there will be a more official migration path.

## Conclusion

In most cases, installation is as simple as getting a pre-built `helm` binary
Expand Down
82 changes: 82 additions & 0 deletions docs/sql-storage.md
@@ -0,0 +1,82 @@
# Store release information in an SQL database

You may be willing to store release information in an SQL database - in
particular, if your releases weigh more than 1MB and therefore [can't be stored in ConfigMaps or Secrets](https://github.com/helm/helm/issues/1413).

We recommend using [PostgreSQL](https://www.postgresql.org/).

This document describes how to deploy `postgres` atop Kubernetes. This being
said, using an out-of-cluster (managed or not) PostreSQL instance is totally
possible as well.

Here's a Kubernetes manifest you can apply to get a minimal PostreSQL pod
running on your Kubernetes cluster. **Don't forget to change the credentials
and, optionally, enable TLS in production deployments**.

```yaml
apiVersion: v1
kind: Service
metadata:
name: tiller-postgres
spec:
ports:
- port: 5432
selector:
app: tiller-postgres
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: tiller-postgres
spec:
serviceName: tiller-postgres
selector:
matchLabels:
app: tiller-postgres
replicas: 1
template:
metadata:
labels:
app: tiller-postgres
spec:
containers:
- name: postgres
image: postgres:11-alpine
imagePullPolicy: Always
ports:
- containerPort: 5432
env:
- name: POSTGRES_DB
value: helm
- name: POSTGRES_USER
value: helm
- name: POSTGRES_PASSWORD
value: changemeforgodssake
resources:
limits:
memory: 128Mi
requests:
cpu: 50m
memory: 128Mi
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: tiller-postgres-data
volumeClaimTemplates:
- metadata:
name: tiller-postgres-data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: default
resources:
requests:
storage: 5Gi
```

Once postgres is deployed, you'll need to install Tiller using `helm init`, with
a few custom CLI flags:

```shell
helm init \
--override \
'spec.template.spec.containers[0].args'='{--storage=sql,--sql-dialect=postgres,--sql-connection-string=postgresql://tiller-postgres:5432/helm?user=helm&password=changemeforgodssake&sslmode=disable}'
```
22 changes: 19 additions & 3 deletions glide.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

141 changes: 69 additions & 72 deletions glide.yaml
@@ -1,74 +1,71 @@
package: k8s.io/helm
import:
- package: golang.org/x/net
subpackages:
- context
- package: golang.org/x/sync
subpackages:
- semaphore
# This is temporary and can probably be removed the next time gRPC is updated
- package: golang.org/x/sys
version: b90733256f2e882e81d52f9126de08df5615afd9
subpackages:
- unix
- windows
- package: github.com/spf13/cobra
version: fe5e611709b0c57fa4a89136deaa8e1d4004d053
- package: github.com/spf13/pflag
version: ~1.0.1
- package: github.com/Masterminds/vcs
# Pin version of mergo that is compatible with both sprig and Kubernetes
- package: github.com/imdario/mergo
version: v0.3.5
- package: github.com/Masterminds/sprig
version: ^2.18.0
- package: github.com/ghodss/yaml
- package: github.com/Masterminds/semver
version: ~1.3.1
- package: github.com/technosophos/moniker
version: ~0.2
- package: github.com/golang/protobuf
version: 1.2.0
subpackages:
- proto
- ptypes/any
- ptypes/timestamp
- package: google.golang.org/grpc
version: 1.18.0
- package: github.com/gosuri/uitable
- package: github.com/asaskevich/govalidator
version: ^4.0.0
- package: golang.org/x/crypto
subpackages:
- openpgp
- ssh/terminal
- package: github.com/gobwas/glob
version: ^0.2.1
- package: github.com/evanphx/json-patch
- package: github.com/BurntSushi/toml
version: ~0.3.0
- package: github.com/prometheus/client_golang
version: 0.8.0
- package: github.com/grpc-ecosystem/go-grpc-prometheus
- package: k8s.io/kubernetes
version: release-1.13
- package: k8s.io/client-go
version: kubernetes-1.13.1
- package: k8s.io/api
version: kubernetes-1.13.1
- package: k8s.io/apimachinery
version: kubernetes-1.13.1
- package: k8s.io/apiserver
version: kubernetes-1.13.1
- package: k8s.io/cli-runtime
version: kubernetes-1.13.1
- package: k8s.io/apiextensions-apiserver
version: kubernetes-1.13.1
- package: github.com/cyphar/filepath-securejoin
version: ^0.2.1

testImports:
- package: github.com/stretchr/testify
version: ^1.1.4
subpackages:
- assert
- package: golang.org/x/net
subpackages:
- context
- package: golang.org/x/sync
subpackages:
- semaphore
- package: golang.org/x/sys
version: b90733256f2e882e81d52f9126de08df5615afd9
subpackages:
- unix
- windows
- package: github.com/spf13/cobra
version: fe5e611709b0c57fa4a89136deaa8e1d4004d053
- package: github.com/spf13/pflag
version: ~1.0.1
- package: github.com/Masterminds/vcs
- package: github.com/imdario/mergo
version: v0.3.5
- package: github.com/Masterminds/sprig
version: ^2.18.0
- package: github.com/ghodss/yaml
- package: github.com/Masterminds/semver
version: ~1.3.1
- package: github.com/technosophos/moniker
version: ~0.2
- package: github.com/golang/protobuf
version: 1.2.0
subpackages:
- proto
- ptypes/any
- ptypes/timestamp
- package: google.golang.org/grpc
version: 1.18.0
- package: github.com/gosuri/uitable
- package: github.com/asaskevich/govalidator
version: ^4.0.0
- package: golang.org/x/crypto
subpackages:
- openpgp
- ssh/terminal
- package: github.com/gobwas/glob
version: ^0.2.1
- package: github.com/evanphx/json-patch
- package: github.com/BurntSushi/toml
version: ~0.3.0
- package: github.com/prometheus/client_golang
version: 0.8.0
- package: github.com/grpc-ecosystem/go-grpc-prometheus
- package: k8s.io/kubernetes
version: release-1.13
- package: k8s.io/client-go
version: kubernetes-1.13.1
- package: k8s.io/api
version: kubernetes-1.13.1
- package: k8s.io/apimachinery
version: kubernetes-1.13.1
- package: k8s.io/apiserver
version: kubernetes-1.13.1
- package: k8s.io/cli-runtime
version: kubernetes-1.13.1
- package: k8s.io/apiextensions-apiserver
version: kubernetes-1.13.1
- package: github.com/cyphar/filepath-securejoin
version: ^0.2.1
- package: github.com/jmoiron/sqlx
version: ^1.2.0
- package: github.com/rubenv/sql-migrate
- package: github.com/DATA-DOG/go-sqlmock
version: ^1.3.2

0 comments on commit 531aaaa

Please sign in to comment.