Skip to content

Commit

Permalink
Add kustomize support for MinIOInstance (#168)
Browse files Browse the repository at this point in the history
Co-authored-by: Daniel Valdivia <hola@danielvaldivia.com>
  • Loading branch information
nitisht and dvaldivia committed Jul 20, 2020
1 parent e651e28 commit accb529
Show file tree
Hide file tree
Showing 21 changed files with 280 additions and 24 deletions.
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ MinIO-Operator brings native MinIO, [MCS](https://github.com/minio/mcs), and [KE
| Feature | Reference Document |
|-------------------------|--------------------|
| Create and delete highly available distributed MinIO clusters | [Create a MinIO Instance](https://github.com/minio/minio-operator#create-a-minio-instance). |
| Automatic TLS for MinIO | [Automatic TLS for MinIO Instance](https://github.com/minio/minio-operator/blob/master/docs/tls.md#automatic-csr-generation). |
| Expand an existing MinIO cluster | [Expand a MinIO Cluster](https://github.com/minio/minio-operator/blob/master/docs/adding-zones.md). |
| Use a custom template for hostname discovery | [Custom Hostname Discovery](https://github.com/minio/minio-operator/blob/master/docs/custom-name-templates.md). |
| TLS Configuration | [TLS for MinIO Instance](https://github.com/minio/minio-operator/blob/master/docs/tls.md). |
| Expand an existing MinIO cluster | [Expand a MinIO Cluster](https://github.com/minio/minio-operator/blob/master/docs/adding-zones.md). |
| Use a custom template for hostname discovery | [Custom Hostname Discovery](https://github.com/minio/minio-operator/blob/master/docs/custom-name-templates.md). |
| Use PodSecurityPolicy for MinIO Pods | [Apply PodSecurityPolicy](https://github.com/minio/minio-operator/blob/master/docs/pod-security-policy.md). |
| Deploy MCS with MinIO cluster | [Deploy MinIO Instance with MCS](https://github.com/minio/minio-operator/blob/master/docs/mcs.md). |
| Deploy KES with MinIO cluster | [Deploy MinIO Instance with KES](https://github.com/minio/minio-operator/blob/master/docs/kes.md). |
Expand All @@ -35,7 +35,6 @@ Advanced users can leverage [kustomize](https://github.com/kubernetes-sigs/kusto

```bash
git clone https://github.com/minio/minio-operator
cd operator-deployment
kustomize build | kubectl apply -f -
```

Expand All @@ -57,8 +56,8 @@ These variables may be passed to operator Deployment in order to modify some of

| Name | Default | Description |
| --- | --- | --- |
| `WATCHED_NAMESPACE` | | If set, the operator will watch only MinIO resources deployed in the specified namespace. All namespaces are watched if empty |
| `CLUSTER_DOMAIN` | cluster.local | Cluster Domain of the Kubernetes cluster |
| `CLUSTER_DOMAIN` | `cluster.local` | Cluster Domain of the Kubernetes cluster |
| `WATCHED_NAMESPACE` | `-` | If set, the operator will watch MinIOInstance resources in specified namespace only. If empty, operator will watch all namespaces. |

## Explore Further

Expand Down
4 changes: 2 additions & 2 deletions examples/minioinstance-kes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
## Secret with credentials to be used by MinIO instance.
Expand Down Expand Up @@ -85,7 +85,7 @@ spec:
storage: 1Ti
## Define configuration for MCS (Graphical user interface for MinIO)
mcs:
image: minio/mcs:v0.1.0
image: minio/mcs:v0.2.0
replicas: 2
mcsSecret:
name: mcs-secret
Expand Down
4 changes: 2 additions & 2 deletions examples/minioinstance-mcs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
## Secret with credentials to be used by MinIO instance.
Expand Down Expand Up @@ -85,7 +85,7 @@ spec:
storage: 1Ti
## Define configuration for MCS (Graphical user interface for MinIO)
mcs:
image: minio/mcs:v0.1.0
image: minio/mcs:v0.2.0
replicas: 2
mcsSecret:
name: mcs-secret
Expand Down
2 changes: 1 addition & 1 deletion examples/minioinstance-pod-security-policy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
## Service account to be used for all the MinIO Pods
Expand Down
2 changes: 1 addition & 1 deletion examples/minioinstance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ spec:
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
## Registry location and Tag to download MinIO Server image
image: minio/minio:RELEASE.2020-07-13T18-09-56Z
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
## A ClusterIP Service will be created with the given name
serviceName: minio-internal-service
zones:
Expand Down
15 changes: 8 additions & 7 deletions operator-deployment/kustomization.yaml → kustomization.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: minio-operator

# Configure number of MinIO Operator Deployment Replicas
replicas:
Expand All @@ -11,7 +10,7 @@ replicas:
images:
- name: minio/k8s-operator
newName: minio/k8s-operator
newTag: 2.0.6
newTag: 2.0.9

# Configure the Cluster Domain and NameSpace to Watch
configMapGenerator:
Expand Down Expand Up @@ -39,6 +38,8 @@ patchesJson6902:
path: /metadata/name
value: "minio-operator"
namespace: minio-operator

vars:
- name: CLUSTER_DOMAIN
objref:
Expand All @@ -56,8 +57,8 @@ vars:
fieldpath: data.WATCHED_NAMESPACE

resources:
- namespace.yaml
- service-account.yaml
- crd.yaml
- rbac.yaml
- deployment.yaml
- operator-kustomize/namespace.yaml
- operator-kustomize/service-account.yaml
- operator-kustomize/crd.yaml
- operator-kustomize/rbac.yaml
- operator-kustomize/deployment.yaml
60 changes: 60 additions & 0 deletions minioinstance-kustomize/kes-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
apiVersion: v1
kind: Secret
metadata:
name: kes-config
type: Opaque
stringData:
server-config.yaml: |-
address: 0.0.0.0:7373
root: _ # Effectively disabled since no root identity necessary.
tls:
key: /home/server.key # Path to the TLS private key
cert: /home/server.crt # Path to the TLS certificate
proxy:
identities: []
header:
cert: X-Tls-Client-Cert
policy:
my-policy:
paths:
- /v1/key/create/*
- /v1/key/generate/*
- /v1/key/decrypt/*
identities:
- ${MINIO_ID}
cache:
expiry:
any: 5m0s
unused: 20s
log:
error: on
audit: off
keys:
fs:
path: "/kes" # Path to directory. Keys will be stored as files. Not Recommended for Production.
# vault:
# endpoint: "" # The Vault endpoint - e.g. https://127.0.0.1:8200
# namespace: "" # An optional Vault namespace. See: https://www.vaultproject.io/docs/enterprise/namespaces/index.html
# prefix: "" # An optional K/V prefix. The server will store keys under this prefix.
# approle: # AppRole credentials. See: https://www.vaultproject.io/docs/auth/approle.html
# id: "" # Your AppRole Role ID
# secret: "" # Your AppRole Secret ID
# retry: 15s # Duration until the server tries to re-authenticate after connection loss.
# tls: # The Vault client TLS configuration for mTLS authentication and certificate verification
# key: "" # Path to the TLS client private key for mTLS authentication to Vault
# cert: "" # Path to the TLS client certificate for mTLS authentication to Vailt
# ca: "" # Path to one or multiple PEM root CA certificates
# status: # Vault status configuration. The server will periodically reach out to Vault to check its status.
# ping: 10s # Duration until the server checks Vault's status again.
# aws:
# # The AWS SecretsManager key store. The server will store
# # secret keys at the AWS SecretsManager encrypted with
# # AWS-KMS. See: https://aws.amazon.com/secrets-manager
# secretsmanager:
# endpoint: "" # The AWS SecretsManager endpoint - e.g.: secretsmanager.us-east-2.amazonaws.com
# region: "" # The AWS region of the SecretsManager - e.g.: us-east-2
# kmskey: "" # The AWS-KMS key ID used to en/decrypt secrets at the SecretsManager. By default (if not set) the default AWS-KMS key will be used.
# credentials: # The AWS credentials for accessing secrets at the AWS SecretsManager.
# accesskey: "" # Your AWS Access Key
# secretkey: "" # Your AWS Secret Key
# token: "" # Your AWS session token (usually optional)
84 changes: 84 additions & 0 deletions minioinstance-kustomize/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: minio
configurations:
- kustomizeconfig/minioinstance.yaml

# Configure tenant name, volumes per server, capacity per volume and storage class
configMapGenerator:
- name: minio-storage
literals:
- tenant="minio-tenant1"
- volumesPerServer=4
- capacity="5Ti"
- storageClass=""
- zones="rack1:4"

# Configure repo and tag of MinIO, MCS & KES Image
images:
- name: minio/minio
newName: minio/minio
newTag: RELEASE.2020-07-14T19-14-30Z
- name: minio/mcs
newName: minio/mcs
newTag: v0.2.0
- name: minio/kes
newName: minio/kes
newTag: v0.10.1

# Configure the Namespace
patchesJson6902:
- target:
version: v1
kind: Namespace
name: minio
patch: |-
- op: replace
path: /metadata/name
value: "minio"
vars:
- name: NAME
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.tenant
- name: ZONES
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.zones
- name: VOLUMES_PER_SERVER
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.volumesPerServer
- name: CAPACITY
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.capacity
- name: STORAGE_CLASS
objref:
kind: ConfigMap
name: minio-storage
apiVersion: v1
fieldref:
fieldpath: data.storageClass


resources:
- namespace.yaml
- mcs-secret.yaml
- kes-secret.yaml
- minio-secret.yaml
- minio-service.yaml
- minio-tenant.yaml
19 changes: 19 additions & 0 deletions minioinstance-kustomize/kustomizeconfig/minioinstance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
images:
- path: spec/mcs/image
kind: MinIOInstance
- path: spec/kes/image
kind: MinIOInstance
- path: spec/image
kind: MinIOInstance

varReference:
- path: metadata/name
kind: MinIOInstance
- path: spec/volumesPerServer
kind: MinIOInstance
- path: spec/volumeClaimTemplate/spec/resources/requests/storage
kind: MinIOInstance
- path: spec/volumeClaimTemplate/spec/storageClassName
kind: MinIOInstance
- path: spec/zones
kind: MinIOInstance
11 changes: 11 additions & 0 deletions minioinstance-kustomize/mcs-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: v1
kind: Secret
metadata:
name: mcs-secret
type: Opaque
data:
MCS_HMAC_JWT_SECRET: WU9VUkpXVFNJR05JTkdTRUNSRVQ= # base 64 encoded "YOURJWTSIGNINGSECRET" (echo -n 'YOURJWTSIGNINGSECRET' | base64)
MCS_PBKDF_PASSPHRASE: U0VDUkVU # base 64 encoded "SECRET" (echo -n 'SECRET' | base64)
MCS_PBKDF_SALT: U0VDUkVU # base 64 encoded "SECRET" (echo -n 'SECRET' | base64)
MCS_ACCESS_KEY: WU9VUk1DU0FDQ0VTUw== # base 64 encoded "YOURMCSACCESS" (echo -n 'YOURMCSACCESS' | base64)
MCS_SECRET_KEY: WU9VUk1DU1NFQ1JFVA== # base 64 encoded "YOURMCSSECRET" (echo -n 'YOURMCSSECRET' | base64)
8 changes: 8 additions & 0 deletions minioinstance-kustomize/minio-secret.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: v1
kind: Secret
metadata:
name: minio-creds-secret
type: Opaque
data:
accesskey: bWluaW8= # base 64 encoded "minio" (echo -n 'minio' | base64)
secretkey: bWluaW8xMjM= # based 64 encoded "minio123" (echo -n 'minio123' | base64)
15 changes: 15 additions & 0 deletions minioinstance-kustomize/minio-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: minio-service
spec:
type: ClusterIP
ports:
- port: 9000
targetPort: 9000
protocol: TCP
# Optional field
# By default and for convenience, the Kubernetes control plane will allocate a port from a range (default: 30000-32767)
# nodePort: 30007
selector:
app: minio
55 changes: 55 additions & 0 deletions minioinstance-kustomize/minio-tenant.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
apiVersion: operator.min.io/v1
kind: MinIOInstance
metadata:
name: $(NAME)
namespace: minio
spec:
metadata:
labels:
app: minio
annotations:
prometheus.io/path: /minio/prometheus/metrics
prometheus.io/port: "9000"
prometheus.io/scrape: "true"
image: minio/minio:RELEASE.2020-07-14T19-14-30Z
serviceName: minio-internal-service
credsSecret:
name: minio-creds-secret
zones: $(ZONES)
volumesPerServer: $(VOLUMES_PER_SERVER)
mountPath: /export
volumeClaimTemplate:
metadata:
name: data
spec:
accessModes:
- ReadWriteOnce
storageClassName: $(STORAGE_CLASS)
resources:
requests:
storage: $(CAPACITY)
mcs:
image: minio/mcs:v0.2.0
replicas: 2
mcsSecret:
name: mcs-secret
metadata:
labels:
app: mcs
kes:
image: minio/kes:v0.10.1
replicas: 2
kesSecret:
name: kes-config
metadata:
labels:
app: kes
requestAutoCert: true
podManagementPolicy: Parallel
certConfig:
commonName: ""
organizationName: []
dnsNames: []
liveness:
initialDelaySeconds: 120
periodSeconds: 60
4 changes: 4 additions & 0 deletions minioinstance-kustomize/namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: minio

0 comments on commit accb529

Please sign in to comment.